2024-04-18T22:56:56.170 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-18T22:56:56.170 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} --owner scheduled_teuthology@teuthology --archive /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678 --name teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi --verbose -- /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/orig.config.yaml 2024-04-18T22:56:56.214 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T22:56:56.313 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678 branch: squid description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} email: ceph-qa@ceph.com first_in_suite: false job_id: '7662678' kernel: kdb: 1 sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: squid ceph: conf: global: mon election default strategy: 3 osd debug inject dispatch delay duration: 0.1 osd debug inject dispatch delay probability: 0.1 osd_async_recovery_min_cost: 1 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon scrub interval: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug reject backfill probability: 0.3 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max backfills: 3 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd scrub max interval: 120 osd scrub min interval: 60 osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - slow request - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: d0bf63dd179c05751e4be49477389aa290c73089 owner: scheduled_teuthology@teuthology priority: 101 repo: https://git.ceph.com/ceph.git roles: - - mon.a - mgr.y - osd.0 - osd.4 - osd.8 - osd.12 - - mon.b - osd.1 - osd.5 - osd.9 - osd.13 - - mon.c - osd.2 - osd.6 - osd.10 - osd.14 - - mgr.x - osd.3 - osd.7 - osd.11 - osd.15 - client.0 seed: 6512 sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 sleep_before_teardown: 0 subset: 28391/100000 suite: rados suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_d0bf63dd179c05751e4be49477389aa290c73089/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: d0bf63dd179c05751e4be49477389aa290c73089 targets: smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGfqB4DVrnrJLzirGB5de2wcaY2pBc8V5fbOxVQlrXlBFgJt0i2bqXUIiqLLPK8FpjoFBa9vi36ZgZIkt+ZHVv8= smithi120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPRCs7ZAWG0V5CWjphJNZmIQrAkvK0fg2glJpBpD5Fp2CFpqQ6vpDVnzHDf4qGqDJTNqf+ie4TgmPzOEYLxr954= smithi136.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBv0kAkKsnr9mYTUL9yt5H8GNLJWUrq7NLlyI7L0+mFDC3b37uB3afLviw4z5pIgEMde9jD1QkY56bSIMSStOrI= smithi142.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWvCQrq9gAqochkHl1mWyigv/lu3X+lFu5BOi/j1f5KDFqtCcpQpXO377WTxoCRP1FN3w8Cn6C35ORnDJvP1QU= tasks: - install: null - ceph: conf: osd: debug monc: 20 - thrashosds: aggressive_pg_num_changes: false chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 min_in: 8 timeout: 1200 - rados: clients: - client.0 ec_pool: true erasure_code_profile: c: 2 crush-failure-domain: osd k: 4 m: 3 name: shecprofile plugin: shec objects: 50 op_weights: append: 100 copy_from: 50 delete: 50 read: 100 rmattr: 25 rollback: 50 setattr: 25 snap_create: 50 snap_remove: 50 write: 0 ops: 400 write_append_excl: false teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-18_21:00:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-18T22:56:56.313 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_d0bf63dd179c05751e4be49477389aa290c73089/qa; will attempt to use it 2024-04-18T22:56:56.314 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_d0bf63dd179c05751e4be49477389aa290c73089/qa/tasks 2024-04-18T22:56:56.314 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-18T22:56:56.315 INFO:teuthology.task.internal:Checking packages... 2024-04-18T22:56:56.340 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '1a6a3e742fd81c6d70be70552499bff767652c32' 2024-04-18T22:56:56.341 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-18T22:56:56.341 INFO:teuthology.packaging:ref: None 2024-04-18T22:56:56.341 INFO:teuthology.packaging:tag: None 2024-04-18T22:56:56.341 INFO:teuthology.packaging:branch: squid 2024-04-18T22:56:56.341 INFO:teuthology.packaging:sha1: 1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T22:56:56.341 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=squid 2024-04-18T22:56:56.540 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2401-gd0bf63dd-1jammy 2024-04-18T22:56:56.541 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-18T22:56:56.548 INFO:teuthology.task.internal:no buildpackages task found 2024-04-18T22:56:56.548 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-18T22:56:56.571 INFO:teuthology.task.internal:Saving configuration 2024-04-18T22:56:56.588 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-18T22:56:56.593 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-18T22:56:56.619 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 22:48:07.207717', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIAejkn2CJS9UlJnvUCNA/qTKMLiAoEwT2snAmB0DESW/'} 2024-04-18T22:56:56.640 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi120.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 22:48:07.208619', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDm4PXXp1iRs6y/N4/ghUlw4nC4GhHQAPlt2EUKverX2'} 2024-04-18T22:56:56.664 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi136.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 22:48:07.205403', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEAnJavyb9naPFoejtKMgHLWsubUYP8EWo/Hn5Zsjv2p'} 2024-04-18T22:56:56.685 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi142.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-18 22:48:07.206810', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPDxLE+kr3snWgX8Y0QogzYYlML/Jb0X11n8+B4dobCI'} 2024-04-18T22:56:56.686 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-18T22:56:56.691 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'] 2024-04-18T22:56:56.691 INFO:teuthology.task.internal:roles: ubuntu@smithi120.front.sepia.ceph.com - ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'] 2024-04-18T22:56:56.692 INFO:teuthology.task.internal:roles: ubuntu@smithi136.front.sepia.ceph.com - ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'] 2024-04-18T22:56:56.692 INFO:teuthology.task.internal:roles: ubuntu@smithi142.front.sepia.ceph.com - ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0'] 2024-04-18T22:56:56.692 INFO:teuthology.run_tasks:Running task console_log... 2024-04-18T22:56:56.823 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb5e75b1280>, signals=[15]) 2024-04-18T22:56:56.823 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-18T22:56:56.830 INFO:teuthology.task.internal:Opening connections... 2024-04-18T22:56:56.830 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T22:56:56.832 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:56:56.912 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T22:56:56.913 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:56:56.990 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T22:56:56.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:56:57.068 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T22:56:57.069 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:56:57.146 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-18T22:56:57.153 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2024-04-18T22:56:57.159 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2024-04-18T22:56:57.159 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Ubuntu" 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="22.04" 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_CODENAME=jammy 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:ID=ubuntu 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE=debian 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-18T22:56:57.208 INFO:teuthology.orchestra.run.smithi073.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-18T22:56:57.209 INFO:teuthology.orchestra.run.smithi073.stdout:UBUNTU_CODENAME=jammy 2024-04-18T22:56:57.209 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2024-04-18T22:56:57.234 DEBUG:teuthology.orchestra.run.smithi120:> uname -m 2024-04-18T22:56:57.240 INFO:teuthology.orchestra.run.smithi120.stdout:x86_64 2024-04-18T22:56:57.240 DEBUG:teuthology.orchestra.run.smithi120:> cat /etc/os-release 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:NAME="Ubuntu" 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_ID="22.04" 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:VERSION_CODENAME=jammy 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:ID=ubuntu 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:ID_LIKE=debian 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-18T22:56:57.288 INFO:teuthology.orchestra.run.smithi120.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-18T22:56:57.289 INFO:teuthology.orchestra.run.smithi120.stdout:UBUNTU_CODENAME=jammy 2024-04-18T22:56:57.289 INFO:teuthology.lock.ops:Updating smithi120.front.sepia.ceph.com on lock server 2024-04-18T22:56:57.317 DEBUG:teuthology.orchestra.run.smithi136:> uname -m 2024-04-18T22:56:57.323 INFO:teuthology.orchestra.run.smithi136.stdout:x86_64 2024-04-18T22:56:57.324 DEBUG:teuthology.orchestra.run.smithi136:> cat /etc/os-release 2024-04-18T22:56:57.369 INFO:teuthology.orchestra.run.smithi136.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-18T22:56:57.369 INFO:teuthology.orchestra.run.smithi136.stdout:NAME="Ubuntu" 2024-04-18T22:56:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_ID="22.04" 2024-04-18T22:56:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-18T22:56:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:VERSION_CODENAME=jammy 2024-04-18T22:56:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:ID=ubuntu 2024-04-18T22:56:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:ID_LIKE=debian 2024-04-18T22:56:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-18T22:56:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-18T22:56:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-18T22:56:57.370 INFO:teuthology.orchestra.run.smithi136.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-18T22:56:57.371 INFO:teuthology.orchestra.run.smithi136.stdout:UBUNTU_CODENAME=jammy 2024-04-18T22:56:57.371 INFO:teuthology.lock.ops:Updating smithi136.front.sepia.ceph.com on lock server 2024-04-18T22:56:57.392 DEBUG:teuthology.orchestra.run.smithi142:> uname -m 2024-04-18T22:56:57.398 INFO:teuthology.orchestra.run.smithi142.stdout:x86_64 2024-04-18T22:56:57.398 DEBUG:teuthology.orchestra.run.smithi142:> cat /etc/os-release 2024-04-18T22:56:57.444 INFO:teuthology.orchestra.run.smithi142.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-04-18T22:56:57.444 INFO:teuthology.orchestra.run.smithi142.stdout:NAME="Ubuntu" 2024-04-18T22:56:57.444 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_ID="22.04" 2024-04-18T22:56:57.444 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-04-18T22:56:57.444 INFO:teuthology.orchestra.run.smithi142.stdout:VERSION_CODENAME=jammy 2024-04-18T22:56:57.445 INFO:teuthology.orchestra.run.smithi142.stdout:ID=ubuntu 2024-04-18T22:56:57.445 INFO:teuthology.orchestra.run.smithi142.stdout:ID_LIKE=debian 2024-04-18T22:56:57.445 INFO:teuthology.orchestra.run.smithi142.stdout:HOME_URL="https://www.ubuntu.com/" 2024-04-18T22:56:57.445 INFO:teuthology.orchestra.run.smithi142.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-04-18T22:56:57.445 INFO:teuthology.orchestra.run.smithi142.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-04-18T22:56:57.445 INFO:teuthology.orchestra.run.smithi142.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-04-18T22:56:57.445 INFO:teuthology.orchestra.run.smithi142.stdout:UBUNTU_CODENAME=jammy 2024-04-18T22:56:57.445 INFO:teuthology.lock.ops:Updating smithi142.front.sepia.ceph.com on lock server 2024-04-18T22:56:57.467 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-18T22:56:57.476 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-18T22:56:57.481 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-18T22:56:57.482 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2024-04-18T22:56:57.484 DEBUG:teuthology.orchestra.run.smithi120:> test '!' -e /home/ubuntu/cephtest 2024-04-18T22:56:57.487 DEBUG:teuthology.orchestra.run.smithi136:> test '!' -e /home/ubuntu/cephtest 2024-04-18T22:56:57.489 DEBUG:teuthology.orchestra.run.smithi142:> test '!' -e /home/ubuntu/cephtest 2024-04-18T22:56:57.494 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-18T22:56:57.500 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-18T22:56:57.500 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2024-04-18T22:56:57.532 DEBUG:teuthology.orchestra.run.smithi120:> test -z $(ls -A /var/lib/ceph) 2024-04-18T22:56:57.535 DEBUG:teuthology.orchestra.run.smithi136:> test -z $(ls -A /var/lib/ceph) 2024-04-18T22:56:57.537 DEBUG:teuthology.orchestra.run.smithi142:> test -z $(ls -A /var/lib/ceph) 2024-04-18T22:56:57.555 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-18T22:56:57.643 INFO:teuthology.run_tasks:Running task kernel... 2024-04-18T22:56:57.658 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-18T22:56:57.658 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}, 'mon.c': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-18T22:56:57.659 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-18T22:56:57.659 DEBUG:teuthology.orchestra.run.smithi120:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-18T22:56:57.660 DEBUG:teuthology.orchestra.run.smithi136:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-18T22:56:57.660 DEBUG:teuthology.orchestra.run.smithi142:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-18T22:56:57.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:56:57.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:56:57.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:56:57.667 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T22:56:57.668 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-04-18T22:56:57.668 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-04-18T22:56:57.668 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-04-18T22:56:57.669 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-04-18T22:56:57.712 INFO:teuthology.orchestra.run.smithi120.stdout:5.15.0-56-generic 2024-04-18T22:56:57.712 INFO:teuthology.task.kernel:Running kernel on smithi120: 5.15.0-56-generic 2024-04-18T22:56:57.712 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get clean 2024-04-18T22:56:57.713 INFO:teuthology.orchestra.run.smithi142.stdout:5.15.0-56-generic 2024-04-18T22:56:57.714 INFO:teuthology.task.kernel:Running kernel on smithi142: 5.15.0-56-generic 2024-04-18T22:56:57.714 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get clean 2024-04-18T22:56:57.715 INFO:teuthology.orchestra.run.smithi073.stdout:5.15.0-56-generic 2024-04-18T22:56:57.715 INFO:teuthology.task.kernel:Running kernel on smithi073: 5.15.0-56-generic 2024-04-18T22:56:57.715 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get clean 2024-04-18T22:56:57.719 INFO:teuthology.orchestra.run.smithi136.stdout:5.15.0-56-generic 2024-04-18T22:56:57.719 INFO:teuthology.task.kernel:Running kernel on smithi136: 5.15.0-56-generic 2024-04-18T22:56:57.719 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2024-04-18T22:56:57.844 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get update 2024-04-18T22:56:57.852 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get update 2024-04-18T22:56:57.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-04-18T22:56:57.867 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get update 2024-04-18T22:56:58.014 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:56:58.036 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:56:58.036 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-18T22:56:58.036 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-18T22:56:58.040 INFO:teuthology.orchestra.run.smithi120.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-18T22:56:58.115 INFO:teuthology.orchestra.run.smithi120.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-18T22:56:58.117 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-18T22:56:58.120 INFO:teuthology.orchestra.run.smithi120.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-18T22:56:58.123 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:56:58.123 INFO:teuthology.orchestra.run.smithi142.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-18T22:56:58.141 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:56:58.142 INFO:teuthology.orchestra.run.smithi073.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-18T22:56:58.177 INFO:teuthology.orchestra.run.smithi142.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-18T22:56:58.206 INFO:teuthology.orchestra.run.smithi073.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-04-18T22:56:58.250 INFO:teuthology.orchestra.run.smithi120.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [612 kB] 2024-04-18T22:56:58.260 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,346 kB] 2024-04-18T22:56:58.331 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [612 kB] 2024-04-18T22:56:58.332 INFO:teuthology.orchestra.run.smithi120.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,572 kB] 2024-04-18T22:56:58.372 INFO:teuthology.orchestra.run.smithi120.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [300 kB] 2024-04-18T22:56:58.375 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-18T22:56:58.378 INFO:teuthology.orchestra.run.smithi120.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-18T22:56:58.379 INFO:teuthology.orchestra.run.smithi120.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-18T22:56:58.380 INFO:teuthology.orchestra.run.smithi120.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,765 kB] 2024-04-18T22:56:58.383 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [237 kB] 2024-04-18T22:56:58.387 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-18T22:56:58.388 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-18T22:56:58.407 INFO:teuthology.orchestra.run.smithi120.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [298 kB] 2024-04-18T22:56:58.411 INFO:teuthology.orchestra.run.smithi120.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:56:58.412 INFO:teuthology.orchestra.run.smithi120.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-18T22:56:58.421 INFO:teuthology.orchestra.run.smithi120.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-18T22:56:58.422 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-18T22:56:58.423 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-18T22:56:58.426 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,572 kB] 2024-04-18T22:56:58.429 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:56:58.429 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [852 kB] 2024-04-18T22:56:58.444 INFO:teuthology.orchestra.run.smithi120.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-18T22:56:58.447 INFO:teuthology.orchestra.run.smithi120.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-18T22:56:58.449 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-18T22:56:58.454 INFO:teuthology.orchestra.run.smithi120.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-18T22:56:58.454 INFO:teuthology.orchestra.run.smithi120.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-18T22:56:58.454 INFO:teuthology.orchestra.run.smithi120.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-18T22:56:58.455 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [163 kB] 2024-04-18T22:56:58.456 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-18T22:56:58.470 INFO:teuthology.orchestra.run.smithi120.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-18T22:56:58.470 INFO:teuthology.orchestra.run.smithi120.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-18T22:56:58.470 INFO:teuthology.orchestra.run.smithi120.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-18T22:56:58.470 INFO:teuthology.orchestra.run.smithi120.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-18T22:56:58.470 INFO:teuthology.orchestra.run.smithi120.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-18T22:56:58.474 INFO:teuthology.orchestra.run.smithi120.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-18T22:56:58.475 INFO:teuthology.orchestra.run.smithi120.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-18T22:56:58.488 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [300 kB] 2024-04-18T22:56:58.488 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-18T22:56:58.489 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-18T22:56:58.489 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-18T22:56:58.489 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-18T22:56:58.493 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-18T22:56:58.494 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-18T22:56:58.495 INFO:teuthology.orchestra.run.smithi120.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-18T22:56:58.496 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,765 kB] 2024-04-18T22:56:58.496 INFO:teuthology.orchestra.run.smithi120.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-18T22:56:58.525 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [298 kB] 2024-04-18T22:56:58.529 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:56:58.529 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-18T22:56:58.533 INFO:teuthology.orchestra.run.smithi142.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-18T22:56:58.537 INFO:teuthology.orchestra.run.smithi142.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-18T22:56:58.538 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-18T22:56:58.549 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-18T22:56:58.551 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-18T22:56:58.551 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-18T22:56:58.552 INFO:teuthology.orchestra.run.smithi120.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,351 kB] 2024-04-18T22:56:58.552 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-18T22:56:58.552 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-18T22:56:58.561 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-18T22:56:58.561 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-18T22:56:58.562 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-18T22:56:58.562 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-18T22:56:58.564 INFO:teuthology.orchestra.run.smithi073.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,351 kB] 2024-04-18T22:56:58.575 INFO:teuthology.orchestra.run.smithi073.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-04-18T22:56:58.581 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-18T22:56:58.582 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-18T22:56:58.582 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-18T22:56:58.582 INFO:teuthology.orchestra.run.smithi136.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-18T22:56:58.583 INFO:teuthology.orchestra.run.smithi136.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-18T22:56:58.670 INFO:teuthology.orchestra.run.smithi142.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [612 kB] 2024-04-18T22:56:58.716 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [612 kB] 2024-04-18T22:56:58.828 INFO:teuthology.orchestra.run.smithi142.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,351 kB] 2024-04-18T22:56:58.890 INFO:teuthology.orchestra.run.smithi142.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,572 kB] 2024-04-18T22:56:58.916 INFO:teuthology.orchestra.run.smithi073.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,572 kB] 2024-04-18T22:56:58.959 INFO:teuthology.orchestra.run.smithi120.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-18T22:56:58.977 INFO:teuthology.orchestra.run.smithi120.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [238 kB] 2024-04-18T22:56:58.986 INFO:teuthology.orchestra.run.smithi120.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-18T22:56:58.987 INFO:teuthology.orchestra.run.smithi120.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-18T22:56:58.989 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [445 kB] 2024-04-18T22:56:58.995 INFO:teuthology.orchestra.run.smithi142.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [238 kB] 2024-04-18T22:56:59.008 INFO:teuthology.orchestra.run.smithi073.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [238 kB] 2024-04-18T22:56:59.009 INFO:teuthology.orchestra.run.smithi142.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-18T22:56:59.009 INFO:teuthology.orchestra.run.smithi142.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-18T22:56:59.017 INFO:teuthology.orchestra.run.smithi073.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-04-18T22:56:59.017 INFO:teuthology.orchestra.run.smithi073.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-18T22:56:59.018 INFO:teuthology.orchestra.run.smithi073.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,670 kB] 2024-04-18T22:56:59.042 INFO:teuthology.orchestra.run.smithi142.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [300 kB] 2024-04-18T22:56:59.055 INFO:teuthology.orchestra.run.smithi142.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-18T22:56:59.057 INFO:teuthology.orchestra.run.smithi142.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,765 kB] 2024-04-18T22:56:59.065 INFO:teuthology.orchestra.run.smithi073.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [300 kB] 2024-04-18T22:56:59.079 INFO:teuthology.orchestra.run.smithi073.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-04-18T22:56:59.080 INFO:teuthology.orchestra.run.smithi073.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-18T22:56:59.082 INFO:teuthology.orchestra.run.smithi120.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-18T22:56:59.082 INFO:teuthology.orchestra.run.smithi073.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,765 kB] 2024-04-18T22:56:59.083 INFO:teuthology.orchestra.run.smithi120.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-18T22:56:59.089 INFO:teuthology.orchestra.run.smithi142.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [34.1 kB] 2024-04-18T22:56:59.090 INFO:teuthology.orchestra.run.smithi142.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-18T22:56:59.094 INFO:teuthology.orchestra.run.smithi120.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:56:59.094 INFO:teuthology.orchestra.run.smithi120.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-18T22:56:59.107 INFO:teuthology.orchestra.run.smithi142.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:56:59.107 INFO:teuthology.orchestra.run.smithi142.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-18T22:56:59.118 INFO:teuthology.orchestra.run.smithi142.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-18T22:56:59.121 INFO:teuthology.orchestra.run.smithi142.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [38.1 kB] 2024-04-18T22:56:59.122 INFO:teuthology.orchestra.run.smithi142.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [298 kB] 2024-04-18T22:56:59.123 INFO:teuthology.orchestra.run.smithi120.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-18T22:56:59.130 INFO:teuthology.orchestra.run.smithi142.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:56:59.138 INFO:teuthology.orchestra.run.smithi073.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [280 kB] 2024-04-18T22:56:59.142 INFO:teuthology.orchestra.run.smithi073.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [298 kB] 2024-04-18T22:56:59.144 INFO:teuthology.orchestra.run.smithi073.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:56:59.144 INFO:teuthology.orchestra.run.smithi073.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [600 kB] 2024-04-18T22:56:59.150 INFO:teuthology.orchestra.run.smithi073.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-04-18T22:56:59.155 INFO:teuthology.orchestra.run.smithi073.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-18T22:56:59.295 INFO:teuthology.orchestra.run.smithi142.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-18T22:56:59.298 INFO:teuthology.orchestra.run.smithi142.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-04-18T22:56:59.300 INFO:teuthology.orchestra.run.smithi120.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-04-18T22:56:59.314 INFO:teuthology.orchestra.run.smithi073.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [699 kB] 2024-04-18T22:56:59.333 INFO:teuthology.orchestra.run.smithi073.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-04-18T22:56:59.707 INFO:teuthology.orchestra.run.smithi142.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-18T22:56:59.714 INFO:teuthology.orchestra.run.smithi142.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-18T22:56:59.714 INFO:teuthology.orchestra.run.smithi120.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-18T22:56:59.720 INFO:teuthology.orchestra.run.smithi120.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-18T22:56:59.720 INFO:teuthology.orchestra.run.smithi120.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-18T22:56:59.727 INFO:teuthology.orchestra.run.smithi142.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-18T22:56:59.728 INFO:teuthology.orchestra.run.smithi142.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-18T22:56:59.730 INFO:teuthology.orchestra.run.smithi142.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-18T22:56:59.734 INFO:teuthology.orchestra.run.smithi120.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-18T22:56:59.736 INFO:teuthology.orchestra.run.smithi073.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-04-18T22:56:59.738 INFO:teuthology.orchestra.run.smithi120.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-18T22:56:59.742 INFO:teuthology.orchestra.run.smithi073.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.2 kB] 2024-04-18T22:56:59.764 INFO:teuthology.orchestra.run.smithi073.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-04-18T22:56:59.764 INFO:teuthology.orchestra.run.smithi073.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,588 B] 2024-04-18T22:56:59.774 INFO:teuthology.orchestra.run.smithi073.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-04-18T22:56:59.889 INFO:teuthology.orchestra.run.smithi142.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-18T22:56:59.892 INFO:teuthology.orchestra.run.smithi073.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,070 kB] 2024-04-18T22:57:00.000 INFO:teuthology.orchestra.run.smithi142.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-18T22:57:00.005 INFO:teuthology.orchestra.run.smithi073.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [244 kB] 2024-04-18T22:57:00.020 INFO:teuthology.orchestra.run.smithi142.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-18T22:57:00.023 INFO:teuthology.orchestra.run.smithi142.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-18T22:57:00.024 INFO:teuthology.orchestra.run.smithi073.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-04-18T22:57:00.024 INFO:teuthology.orchestra.run.smithi142.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-18T22:57:00.024 INFO:teuthology.orchestra.run.smithi073.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.7 kB] 2024-04-18T22:57:00.025 INFO:teuthology.orchestra.run.smithi142.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-18T22:57:00.026 INFO:teuthology.orchestra.run.smithi142.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-18T22:57:00.026 INFO:teuthology.orchestra.run.smithi142.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-18T22:57:00.028 INFO:teuthology.orchestra.run.smithi073.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,472 B] 2024-04-18T22:57:00.028 INFO:teuthology.orchestra.run.smithi073.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.4 kB] 2024-04-18T22:57:00.029 INFO:teuthology.orchestra.run.smithi073.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-04-18T22:57:00.029 INFO:teuthology.orchestra.run.smithi073.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-18T22:57:00.032 INFO:teuthology.orchestra.run.smithi142.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [59.2 kB] 2024-04-18T22:57:00.033 INFO:teuthology.orchestra.run.smithi073.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [67.1 kB] 2024-04-18T22:57:00.035 INFO:teuthology.orchestra.run.smithi142.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-18T22:57:00.037 INFO:teuthology.orchestra.run.smithi073.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [11.0 kB] 2024-04-18T22:57:00.199 INFO:teuthology.orchestra.run.smithi142.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-18T22:57:00.203 INFO:teuthology.orchestra.run.smithi073.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-04-18T22:57:00.282 INFO:teuthology.orchestra.run.smithi142.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-18T22:57:00.285 INFO:teuthology.orchestra.run.smithi073.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [17.2 kB] 2024-04-18T22:57:00.364 INFO:teuthology.orchestra.run.smithi142.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-18T22:57:00.372 INFO:teuthology.orchestra.run.smithi073.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [28.4 kB] 2024-04-18T22:57:00.445 INFO:teuthology.orchestra.run.smithi142.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-18T22:57:00.451 INFO:teuthology.orchestra.run.smithi073.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.2 kB] 2024-04-18T22:57:00.475 INFO:teuthology.orchestra.run.smithi142.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-18T22:57:00.484 INFO:teuthology.orchestra.run.smithi073.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-04-18T22:57:04.056 INFO:teuthology.orchestra.run.smithi120.stdout:Fetched 12.9 MB in 2s (7,167 kB/s) 2024-04-18T22:57:04.078 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 12.9 MB in 2s (7,094 kB/s) 2024-04-18T22:57:04.690 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 12.9 MB in 3s (5,073 kB/s) 2024-04-18T22:57:04.848 INFO:teuthology.orchestra.run.smithi142.stdout:Fetched 12.9 MB in 3s (5,051 kB/s) 2024-04-18T22:57:05.428 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-04-18T22:57:05.445 DEBUG:teuthology.orchestra.run.smithi120:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:57:05.451 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-04-18T22:57:05.467 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:57:05.503 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-04-18T22:57:05.527 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-04-18T22:57:05.701 INFO:teuthology.orchestra.run.smithi120.stdout:Building dependency tree... 2024-04-18T22:57:05.701 INFO:teuthology.orchestra.run.smithi120.stdout:Reading state information... 2024-04-18T22:57:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-04-18T22:57:05.727 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-04-18T22:57:05.855 INFO:teuthology.orchestra.run.smithi120.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:57:05.856 INFO:teuthology.orchestra.run.smithi120.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:57:05.856 INFO:teuthology.orchestra.run.smithi120.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:57:05.856 INFO:teuthology.orchestra.run.smithi120.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:57:05.856 INFO:teuthology.orchestra.run.smithi120.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-18T22:57:05.856 INFO:teuthology.orchestra.run.smithi120.stdout: python2.7-minimal 2024-04-18T22:57:05.856 INFO:teuthology.orchestra.run.smithi120.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:57:05.857 INFO:teuthology.orchestra.run.smithi120.stdout:The following additional packages will be installed: 2024-04-18T22:57:05.858 INFO:teuthology.orchestra.run.smithi120.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-18T22:57:05.858 INFO:teuthology.orchestra.run.smithi120.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-18T22:57:05.858 INFO:teuthology.orchestra.run.smithi120.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-18T22:57:05.859 INFO:teuthology.orchestra.run.smithi120.stdout:Suggested packages: 2024-04-18T22:57:05.859 INFO:teuthology.orchestra.run.smithi120.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-18T22:57:05.859 INFO:teuthology.orchestra.run.smithi120.stdout:Recommended packages: 2024-04-18T22:57:05.859 INFO:teuthology.orchestra.run.smithi120.stdout: thermald 2024-04-18T22:57:05.893 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:57:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:57:05.893 INFO:teuthology.orchestra.run.smithi136.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:57:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:57:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-18T22:57:05.894 INFO:teuthology.orchestra.run.smithi136.stdout: python2.7-minimal 2024-04-18T22:57:05.894 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:57:05.895 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-04-18T22:57:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-18T22:57:05.895 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-18T22:57:05.896 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-18T22:57:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-04-18T22:57:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-18T22:57:05.897 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2024-04-18T22:57:05.897 INFO:teuthology.orchestra.run.smithi136.stdout: thermald 2024-04-18T22:57:05.897 INFO:teuthology.orchestra.run.smithi120.stdout:The following NEW packages will be installed: 2024-04-18T22:57:05.898 INFO:teuthology.orchestra.run.smithi120.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-18T22:57:05.898 INFO:teuthology.orchestra.run.smithi120.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-18T22:57:05.898 INFO:teuthology.orchestra.run.smithi120.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-18T22:57:05.898 INFO:teuthology.orchestra.run.smithi120.stdout:The following packages will be upgraded: 2024-04-18T22:57:05.899 INFO:teuthology.orchestra.run.smithi120.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-18T22:57:05.931 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-04-18T22:57:05.946 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-04-18T22:57:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-18T22:57:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-18T22:57:05.946 INFO:teuthology.orchestra.run.smithi136.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-18T22:57:05.947 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages will be upgraded: 2024-04-18T22:57:05.948 INFO:teuthology.orchestra.run.smithi136.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-18T22:57:05.949 DEBUG:teuthology.orchestra.run.smithi073:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:57:06.006 INFO:teuthology.orchestra.run.smithi136.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:57:06.006 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 113 MB of archives. 2024-04-18T22:57:06.006 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-18T22:57:06.006 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-18T22:57:06.009 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-04-18T22:57:06.078 INFO:teuthology.orchestra.run.smithi120.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:57:06.078 INFO:teuthology.orchestra.run.smithi120.stdout:Need to get 113 MB of archives. 2024-04-18T22:57:06.078 INFO:teuthology.orchestra.run.smithi120.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-18T22:57:06.079 INFO:teuthology.orchestra.run.smithi120.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-18T22:57:06.130 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-04-18T22:57:06.147 DEBUG:teuthology.orchestra.run.smithi142:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:57:06.197 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-04-18T22:57:06.210 INFO:teuthology.orchestra.run.smithi073.stdout:Building dependency tree... 2024-04-18T22:57:06.210 INFO:teuthology.orchestra.run.smithi073.stdout:Reading state information... 2024-04-18T22:57:06.374 INFO:teuthology.orchestra.run.smithi142.stdout:Building dependency tree... 2024-04-18T22:57:06.374 INFO:teuthology.orchestra.run.smithi142.stdout:Reading state information... 2024-04-18T22:57:06.387 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:57:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:57:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:57:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:57:06.387 INFO:teuthology.orchestra.run.smithi073.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-18T22:57:06.388 INFO:teuthology.orchestra.run.smithi073.stdout: python2.7-minimal 2024-04-18T22:57:06.388 INFO:teuthology.orchestra.run.smithi073.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:57:06.389 INFO:teuthology.orchestra.run.smithi073.stdout:The following additional packages will be installed: 2024-04-18T22:57:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-18T22:57:06.389 INFO:teuthology.orchestra.run.smithi073.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-18T22:57:06.390 INFO:teuthology.orchestra.run.smithi073.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-18T22:57:06.391 INFO:teuthology.orchestra.run.smithi073.stdout:Suggested packages: 2024-04-18T22:57:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-18T22:57:06.391 INFO:teuthology.orchestra.run.smithi073.stdout:Recommended packages: 2024-04-18T22:57:06.391 INFO:teuthology.orchestra.run.smithi073.stdout: thermald 2024-04-18T22:57:06.444 INFO:teuthology.orchestra.run.smithi073.stdout:The following NEW packages will be installed: 2024-04-18T22:57:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-18T22:57:06.444 INFO:teuthology.orchestra.run.smithi073.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-18T22:57:06.445 INFO:teuthology.orchestra.run.smithi073.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-18T22:57:06.446 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages will be upgraded: 2024-04-18T22:57:06.446 INFO:teuthology.orchestra.run.smithi073.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-18T22:57:06.483 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-18T22:57:06.526 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:57:06.526 INFO:teuthology.orchestra.run.smithi142.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:57:06.526 INFO:teuthology.orchestra.run.smithi142.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:57:06.527 INFO:teuthology.orchestra.run.smithi142.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:57:06.527 INFO:teuthology.orchestra.run.smithi142.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-04-18T22:57:06.527 INFO:teuthology.orchestra.run.smithi142.stdout: python2.7-minimal 2024-04-18T22:57:06.527 INFO:teuthology.orchestra.run.smithi142.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:57:06.528 INFO:teuthology.orchestra.run.smithi142.stdout:The following additional packages will be installed: 2024-04-18T22:57:06.528 INFO:teuthology.orchestra.run.smithi142.stdout: linux-generic linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-18T22:57:06.528 INFO:teuthology.orchestra.run.smithi142.stdout: linux-headers-generic linux-image-5.15.0-105-generic 2024-04-18T22:57:06.529 INFO:teuthology.orchestra.run.smithi142.stdout: linux-modules-5.15.0-105-generic linux-modules-extra-5.15.0-105-generic 2024-04-18T22:57:06.530 INFO:teuthology.orchestra.run.smithi142.stdout:Suggested packages: 2024-04-18T22:57:06.530 INFO:teuthology.orchestra.run.smithi142.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-04-18T22:57:06.530 INFO:teuthology.orchestra.run.smithi142.stdout:Recommended packages: 2024-04-18T22:57:06.530 INFO:teuthology.orchestra.run.smithi142.stdout: thermald 2024-04-18T22:57:06.583 INFO:teuthology.orchestra.run.smithi142.stdout:The following NEW packages will be installed: 2024-04-18T22:57:06.584 INFO:teuthology.orchestra.run.smithi142.stdout: linux-headers-5.15.0-105 linux-headers-5.15.0-105-generic 2024-04-18T22:57:06.584 INFO:teuthology.orchestra.run.smithi142.stdout: linux-image-5.15.0-105-generic linux-modules-5.15.0-105-generic 2024-04-18T22:57:06.584 INFO:teuthology.orchestra.run.smithi142.stdout: linux-modules-extra-5.15.0-105-generic 2024-04-18T22:57:06.585 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages will be upgraded: 2024-04-18T22:57:06.585 INFO:teuthology.orchestra.run.smithi142.stdout: linux-generic linux-headers-generic linux-image-generic 2024-04-18T22:57:06.588 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-18T22:57:06.763 INFO:teuthology.orchestra.run.smithi142.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:57:06.763 INFO:teuthology.orchestra.run.smithi142.stdout:Need to get 113 MB of archives. 2024-04-18T22:57:06.763 INFO:teuthology.orchestra.run.smithi142.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-18T22:57:06.764 INFO:teuthology.orchestra.run.smithi142.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-18T22:57:07.476 INFO:teuthology.orchestra.run.smithi120.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-18T22:57:07.490 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-18T22:57:07.490 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-18T22:57:07.490 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-18T22:57:07.632 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-18T22:57:07.659 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-18T22:57:07.830 INFO:teuthology.orchestra.run.smithi120.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-18T22:57:07.842 INFO:teuthology.orchestra.run.smithi073.stdout:3 upgraded, 5 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:57:07.842 INFO:teuthology.orchestra.run.smithi073.stdout:Need to get 113 MB of archives. 2024-04-18T22:57:07.843 INFO:teuthology.orchestra.run.smithi073.stdout:After this operation, 583 MB of additional disk space will be used. 2024-04-18T22:57:07.843 INFO:teuthology.orchestra.run.smithi073.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-105-generic amd64 5.15.0-105.115 [22.7 MB] 2024-04-18T22:57:08.123 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 113 MB in 2s (63.9 MB/s) 2024-04-18T22:57:08.330 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-18T22:57:08.417 INFO:teuthology.orchestra.run.smithi142.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-18T22:57:08.771 INFO:teuthology.orchestra.run.smithi142.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-18T22:57:09.447 INFO:teuthology.orchestra.run.smithi073.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-105-generic amd64 5.15.0-105.115 [11.5 MB] 2024-04-18T22:57:09.799 INFO:teuthology.orchestra.run.smithi073.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-105-generic amd64 5.15.0-105.115 [63.9 MB] 2024-04-18T22:57:10.430 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-18T22:57:10.437 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:10.661 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:10.986 INFO:teuthology.orchestra.run.smithi142.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-18T22:57:10.986 INFO:teuthology.orchestra.run.smithi142.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-18T22:57:10.986 INFO:teuthology.orchestra.run.smithi142.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-18T22:57:11.796 INFO:teuthology.orchestra.run.smithi142.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-18T22:57:11.863 INFO:teuthology.orchestra.run.smithi142.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-18T22:57:12.101 INFO:teuthology.orchestra.run.smithi073.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-18T22:57:12.101 INFO:teuthology.orchestra.run.smithi073.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-18T22:57:12.101 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-18T22:57:12.240 INFO:teuthology.orchestra.run.smithi142.stdout:Fetched 113 MB in 5s (21.5 MB/s) 2024-04-18T22:57:12.430 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-18T22:57:12.820 INFO:teuthology.orchestra.run.smithi073.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-18T22:57:12.877 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-18T22:57:13.155 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-18T22:57:13.177 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:13.265 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:13.294 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 113 MB in 6s (17.6 MB/s) 2024-04-18T22:57:13.469 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-18T22:57:13.734 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-18T22:57:13.754 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:13.792 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:14.337 INFO:teuthology.orchestra.run.smithi142.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-18T22:57:14.347 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:14.426 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:15.525 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-18T22:57:15.529 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:15.543 INFO:teuthology.orchestra.run.smithi120.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.105.102 [1,694 B] 2024-04-18T22:57:15.543 INFO:teuthology.orchestra.run.smithi120.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.105.102 [2,530 B] 2024-04-18T22:57:15.544 INFO:teuthology.orchestra.run.smithi120.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105 all 5.15.0-105.115 [12.3 MB] 2024-04-18T22:57:15.675 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:16.344 INFO:teuthology.orchestra.run.smithi120.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-105-generic amd64 5.15.0-105.115 [2,857 kB] 2024-04-18T22:57:16.474 INFO:teuthology.orchestra.run.smithi120.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.105.102 [2,382 B] 2024-04-18T22:57:16.882 INFO:teuthology.orchestra.run.smithi120.stdout:Fetched 113 MB in 11s (10.7 MB/s) 2024-04-18T22:57:16.893 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-18T22:57:16.914 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:16.989 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:17.082 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-modules-5.15.0-105-generic. 2024-04-18T22:57:17.563 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-18T22:57:17.583 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:17.620 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:17.861 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-18T22:57:17.882 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:17.969 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:18.469 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-18T22:57:18.490 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:18.622 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:19.076 INFO:teuthology.orchestra.run.smithi120.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-04-18T22:57:19.080 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../0-linux-modules-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:19.228 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:20.380 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:20.480 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:20.839 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:21.001 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:21.312 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-18T22:57:21.323 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-18T22:57:21.378 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-18T22:57:21.921 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-image-5.15.0-105-generic. 2024-04-18T22:57:21.942 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../1-linux-image-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:22.031 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:22.532 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-105-generic. 2024-04-18T22:57:22.552 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:22.590 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:24.116 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:24.235 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:24.559 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:24.704 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:24.999 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-18T22:57:25.021 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-18T22:57:25.028 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:25.114 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-18T22:57:25.136 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:25.453 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:25.581 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:25.892 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-18T22:57:25.915 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-18T22:57:25.959 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-18T22:57:27.194 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-18T22:57:27.231 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:27.294 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:29.692 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../3-linux-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:29.802 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:30.016 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:30.141 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:30.160 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:30.340 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-image-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:30.499 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-18T22:57:30.618 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-headers-5.15.0-105. 2024-04-18T22:57:30.621 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:30.640 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../5-linux-headers-5.15.0-105_5.15.0-105.115_all.deb ... 2024-04-18T22:57:30.694 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-18T22:57:30.701 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-18T22:57:30.730 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:30.760 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:30.772 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-18T22:57:30.901 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:31.878 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-18T22:57:31.897 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:31.936 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:32.229 INFO:teuthology.orchestra.run.smithi136.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:57:32.229 INFO:teuthology.orchestra.run.smithi136.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-18T22:57:32.229 INFO:teuthology.orchestra.run.smithi136.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:57:32.229 INFO:teuthology.orchestra.run.smithi136.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:32.403 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:33.444 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:34.290 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:34.303 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:34.644 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-18T22:57:34.766 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:34.891 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-18T22:57:35.045 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:35.376 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:35.498 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:35.676 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-18T22:57:35.801 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-18T22:57:35.848 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-18T22:57:35.944 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:35.953 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:35.999 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-18T22:57:35.999 INFO:teuthology.orchestra.run.smithi136.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:36.062 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-18T22:57:36.182 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:36.383 INFO:teuthology.orchestra.run.smithi142.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:57:36.383 INFO:teuthology.orchestra.run.smithi142.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-18T22:57:36.383 INFO:teuthology.orchestra.run.smithi142.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:57:36.383 INFO:teuthology.orchestra.run.smithi142.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:36.537 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:37.510 INFO:teuthology.orchestra.run.smithi073.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:57:37.510 INFO:teuthology.orchestra.run.smithi073.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-18T22:57:37.510 INFO:teuthology.orchestra.run.smithi073.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:57:37.510 INFO:teuthology.orchestra.run.smithi073.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:37.645 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package linux-headers-5.15.0-105-generic. 2024-04-18T22:57:37.668 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:37.682 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../6-linux-headers-5.15.0-105-generic_5.15.0-105.115_amd64.deb ... 2024-04-18T22:57:37.711 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:37.948 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:39.062 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:39.424 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-18T22:57:40.208 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-18T22:57:40.334 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:40.397 INFO:teuthology.orchestra.run.smithi142.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-18T22:57:40.397 INFO:teuthology.orchestra.run.smithi142.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:40.446 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-18T22:57:40.563 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-18T22:57:40.665 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:40.729 INFO:teuthology.orchestra.run.smithi073.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-18T22:57:40.730 INFO:teuthology.orchestra.run.smithi073.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:41.411 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.105.102_amd64.deb ... 2024-04-18T22:57:41.537 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking linux-headers-generic (5.15.0.105.102) over (5.15.0.56.54) ... 2024-04-18T22:57:42.304 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-headers-5.15.0-105 (5.15.0-105.115) ... 2024-04-18T22:57:42.459 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-headers-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:42.598 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-headers-generic (5.15.0.105.102) ... 2024-04-18T22:57:42.730 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:44.080 INFO:teuthology.orchestra.run.smithi120.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:57:44.080 INFO:teuthology.orchestra.run.smithi120.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-04-18T22:57:44.080 INFO:teuthology.orchestra.run.smithi120.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:57:44.080 INFO:teuthology.orchestra.run.smithi120.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:44.266 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-modules-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:45.694 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-modules-extra-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:47.111 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-image-generic (5.15.0.105.102) ... 2024-04-18T22:57:47.237 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up linux-generic (5.15.0.105.102) ... 2024-04-18T22:57:47.382 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for linux-image-5.15.0-105-generic (5.15.0-105.115) ... 2024-04-18T22:57:47.435 INFO:teuthology.orchestra.run.smithi120.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-04-18T22:57:47.435 INFO:teuthology.orchestra.run.smithi120.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:50.910 INFO:teuthology.orchestra.run.smithi136.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-18T22:57:50.910 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub' 2024-04-18T22:57:50.936 INFO:teuthology.orchestra.run.smithi136.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:57:50.959 INFO:teuthology.orchestra.run.smithi136.stdout:Generating grub configuration file ... 2024-04-18T22:57:51.282 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:57:51.299 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:51.637 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:57:51.643 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:57:51.705 INFO:teuthology.orchestra.run.smithi136.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:57:51.710 INFO:teuthology.orchestra.run.smithi136.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:57:51.890 INFO:teuthology.orchestra.run.smithi136.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:57:51.890 INFO:teuthology.orchestra.run.smithi136.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:57:51.890 INFO:teuthology.orchestra.run.smithi136.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:57:51.923 INFO:teuthology.orchestra.run.smithi136.stdout:done 2024-04-18T22:57:52.403 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2024-04-18T22:57:52.423 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2024-04-18T22:57:52.423 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2024-04-18T22:57:52.423 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 21 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.15.0.105.102 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:57:52.424 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2024-04-18T22:57:52.425 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-18T22:57:52.425 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-18T22:57:52.425 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-18T22:57:52.425 INFO:teuthology.task.kernel:Installing distro kernel on mon.c... 2024-04-18T22:57:52.425 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi136.front.sepia.ceph.com, path=None, version=distro) 2024-04-18T22:57:52.425 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get clean 2024-04-18T22:57:52.652 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-04-18T22:57:52.807 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:57:52.818 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:57:52.838 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:57:52.897 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-04-18T22:57:53.345 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,351 kB] 2024-04-18T22:57:53.762 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [238 kB] 2024-04-18T22:57:53.771 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [848 kB] 2024-04-18T22:57:53.805 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [162 kB] 2024-04-18T22:57:54.097 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 2,710 kB in 1s (2,115 kB/s) 2024-04-18T22:57:54.108 INFO:teuthology.orchestra.run.smithi142.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-18T22:57:54.108 INFO:teuthology.orchestra.run.smithi142.stdout:Sourcing file `/etc/default/grub' 2024-04-18T22:57:54.118 INFO:teuthology.orchestra.run.smithi142.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:57:54.134 INFO:teuthology.orchestra.run.smithi142.stdout:Generating grub configuration file ... 2024-04-18T22:57:54.401 INFO:teuthology.orchestra.run.smithi142.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:57:54.418 INFO:teuthology.orchestra.run.smithi142.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:54.728 INFO:teuthology.orchestra.run.smithi142.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:57:54.733 INFO:teuthology.orchestra.run.smithi142.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:57:54.795 INFO:teuthology.orchestra.run.smithi142.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:57:54.800 INFO:teuthology.orchestra.run.smithi142.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:57:54.975 INFO:teuthology.orchestra.run.smithi142.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:57:54.975 INFO:teuthology.orchestra.run.smithi142.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:57:54.975 INFO:teuthology.orchestra.run.smithi142.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:57:55.015 INFO:teuthology.orchestra.run.smithi142.stdout:done 2024-04-18T22:57:55.046 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-04-18T22:57:55.063 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:57:55.103 INFO:teuthology.orchestra.run.smithi073.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-18T22:57:55.104 INFO:teuthology.orchestra.run.smithi073.stdout:Sourcing file `/etc/default/grub' 2024-04-18T22:57:55.114 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-04-18T22:57:55.117 INFO:teuthology.orchestra.run.smithi073.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:57:55.142 INFO:teuthology.orchestra.run.smithi073.stdout:Generating grub configuration file ... 2024-04-18T22:57:55.292 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-04-18T22:57:55.292 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-04-18T22:57:55.446 INFO:teuthology.orchestra.run.smithi136.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-18T22:57:55.447 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:57:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:57:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:57:55.447 INFO:teuthology.orchestra.run.smithi136.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:57:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:57:55.448 INFO:teuthology.orchestra.run.smithi136.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:57:55.448 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:57:55.474 INFO:teuthology.orchestra.run.smithi073.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:57:55.478 DEBUG:teuthology.orchestra.run.smithi142:> dpkg -s linux-image-generic 2024-04-18T22:57:55.485 INFO:teuthology.orchestra.run.smithi136.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:57:55.487 DEBUG:teuthology.orchestra.run.smithi136:> dpkg -s linux-image-generic 2024-04-18T22:57:55.495 INFO:teuthology.orchestra.run.smithi073.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:55.497 INFO:teuthology.orchestra.run.smithi142.stdout:Package: linux-image-generic 2024-04-18T22:57:55.497 INFO:teuthology.orchestra.run.smithi142.stdout:Status: install ok installed 2024-04-18T22:57:55.497 INFO:teuthology.orchestra.run.smithi142.stdout:Priority: optional 2024-04-18T22:57:55.497 INFO:teuthology.orchestra.run.smithi142.stdout:Section: kernel 2024-04-18T22:57:55.497 INFO:teuthology.orchestra.run.smithi142.stdout:Installed-Size: 21 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: amd64 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout:Source: linux-meta 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout:Version: 5.15.0.105.102 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout:Recommends: thermald 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout:Description: Generic Linux kernel image 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:57:55.498 INFO:teuthology.orchestra.run.smithi142.stdout: available. 2024-04-18T22:57:55.498 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-18T22:57:55.498 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-18T22:57:55.498 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-18T22:57:55.499 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2024-04-18T22:57:55.499 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi142.front.sepia.ceph.com, path=None, version=distro) 2024-04-18T22:57:55.499 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get clean 2024-04-18T22:57:55.543 INFO:teuthology.orchestra.run.smithi136.stdout:Package: linux-image-generic 2024-04-18T22:57:55.543 INFO:teuthology.orchestra.run.smithi136.stdout:Status: install ok installed 2024-04-18T22:57:55.543 INFO:teuthology.orchestra.run.smithi136.stdout:Priority: optional 2024-04-18T22:57:55.543 INFO:teuthology.orchestra.run.smithi136.stdout:Section: kernel 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout:Installed-Size: 21 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout:Architecture: amd64 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout:Source: linux-meta 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout:Version: 5.15.0.105.102 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout:Recommends: thermald 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout:Description: Generic Linux kernel image 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:57:55.544 INFO:teuthology.orchestra.run.smithi136.stdout: available. 2024-04-18T22:57:55.545 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-04-18T22:57:55.588 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.Jqb4H8pdSw 2024-04-18T22:57:55.588 DEBUG:teuthology.orchestra.run.smithi136:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Jqb4H8pdSw 2024-04-18T22:57:55.645 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.Jqb4H8pdSw 2024-04-18T22:57:55.688 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get update 2024-04-18T22:57:55.773 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.Jqb4H8pdSw is 10KB 2024-04-18T22:57:55.787 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.Jqb4H8pdSw 2024-04-18T22:57:55.792 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-18T22:57:55.804 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:57:55.804 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:57:55.816 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:57:55.836 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:57:55.853 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T22:57:55.853 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-18T22:57:55.853 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-18T22:57:55.875 INFO:teuthology.orchestra.run.smithi073.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:57:55.881 INFO:teuthology.orchestra.run.smithi073.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:57:55.942 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-18T22:57:55.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo update-grub 2024-04-18T22:57:55.952 INFO:teuthology.orchestra.run.smithi073.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:57:55.958 INFO:teuthology.orchestra.run.smithi073.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:57:56.118 INFO:teuthology.orchestra.run.smithi073.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:57:56.118 INFO:teuthology.orchestra.run.smithi073.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:57:56.118 INFO:teuthology.orchestra.run.smithi073.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:57:56.154 INFO:teuthology.orchestra.run.smithi073.stdout:done 2024-04-18T22:57:56.610 DEBUG:teuthology.orchestra.run.smithi073:> dpkg -s linux-image-generic 2024-04-18T22:57:56.633 INFO:teuthology.orchestra.run.smithi073.stdout:Package: linux-image-generic 2024-04-18T22:57:56.633 INFO:teuthology.orchestra.run.smithi073.stdout:Status: install ok installed 2024-04-18T22:57:56.633 INFO:teuthology.orchestra.run.smithi073.stdout:Priority: optional 2024-04-18T22:57:56.633 INFO:teuthology.orchestra.run.smithi073.stdout:Section: kernel 2024-04-18T22:57:56.633 INFO:teuthology.orchestra.run.smithi073.stdout:Installed-Size: 21 2024-04-18T22:57:56.633 INFO:teuthology.orchestra.run.smithi073.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:57:56.634 INFO:teuthology.orchestra.run.smithi073.stdout:Architecture: amd64 2024-04-18T22:57:56.634 INFO:teuthology.orchestra.run.smithi073.stdout:Source: linux-meta 2024-04-18T22:57:56.634 INFO:teuthology.orchestra.run.smithi073.stdout:Version: 5.15.0.105.102 2024-04-18T22:57:56.634 INFO:teuthology.orchestra.run.smithi073.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:57:56.634 INFO:teuthology.orchestra.run.smithi073.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:57:56.634 INFO:teuthology.orchestra.run.smithi073.stdout:Recommends: thermald 2024-04-18T22:57:56.634 INFO:teuthology.orchestra.run.smithi073.stdout:Description: Generic Linux kernel image 2024-04-18T22:57:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:57:56.634 INFO:teuthology.orchestra.run.smithi073.stdout: available. 2024-04-18T22:57:56.634 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-18T22:57:56.634 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-18T22:57:56.634 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-18T22:57:56.635 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-04-18T22:57:56.635 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi073.front.sepia.ceph.com, path=None, version=distro) 2024-04-18T22:57:56.635 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get clean 2024-04-18T22:57:56.805 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get update 2024-04-18T22:57:57.058 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:57:57.058 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:57:57.090 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-04-18T22:57:57.107 DEBUG:teuthology.orchestra.run.smithi142:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:57:57.128 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:57:57.159 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-04-18T22:57:57.210 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:57:57.343 INFO:teuthology.orchestra.run.smithi142.stdout:Building dependency tree... 2024-04-18T22:57:57.343 INFO:teuthology.orchestra.run.smithi142.stdout:Reading state information... 2024-04-18T22:57:57.441 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub' 2024-04-18T22:57:57.449 INFO:teuthology.orchestra.run.smithi136.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:57:57.451 INFO:teuthology.orchestra.run.smithi136.stderr:Generating grub configuration file ... 2024-04-18T22:57:57.499 INFO:teuthology.orchestra.run.smithi142.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-18T22:57:57.500 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:57:57.500 INFO:teuthology.orchestra.run.smithi142.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:57:57.500 INFO:teuthology.orchestra.run.smithi142.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:57:57.500 INFO:teuthology.orchestra.run.smithi142.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:57:57.500 INFO:teuthology.orchestra.run.smithi142.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:57:57.500 INFO:teuthology.orchestra.run.smithi142.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:57:57.501 INFO:teuthology.orchestra.run.smithi142.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:57:57.541 INFO:teuthology.orchestra.run.smithi142.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:57:57.542 DEBUG:teuthology.orchestra.run.smithi142:> dpkg -s linux-image-generic 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Package: linux-image-generic 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Status: install ok installed 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Priority: optional 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Section: kernel 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Installed-Size: 21 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Architecture: amd64 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Source: linux-meta 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Version: 5.15.0.105.102 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:57:57.598 INFO:teuthology.orchestra.run.smithi142.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:57:57.599 INFO:teuthology.orchestra.run.smithi142.stdout:Recommends: thermald 2024-04-18T22:57:57.599 INFO:teuthology.orchestra.run.smithi142.stdout:Description: Generic Linux kernel image 2024-04-18T22:57:57.599 INFO:teuthology.orchestra.run.smithi142.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:57:57.599 INFO:teuthology.orchestra.run.smithi142.stdout: available. 2024-04-18T22:57:57.599 DEBUG:teuthology.orchestra.run.smithi142:> mktemp 2024-04-18T22:57:57.643 INFO:teuthology.orchestra.run.smithi142.stdout:/tmp/tmp.d6UPq5t9Ru 2024-04-18T22:57:57.644 DEBUG:teuthology.orchestra.run.smithi142:> sudo cp /boot/grub/grub.cfg /tmp/tmp.d6UPq5t9Ru 2024-04-18T22:57:57.669 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:57:57.675 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:57.700 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0666 /tmp/tmp.d6UPq5t9Ru 2024-04-18T22:57:57.811 DEBUG:teuthology.orchestra.remote:smithi142:/tmp/tmp.d6UPq5t9Ru is 10KB 2024-04-18T22:57:57.824 DEBUG:teuthology.orchestra.run.smithi142:> rm -fr /tmp/tmp.d6UPq5t9Ru 2024-04-18T22:57:57.829 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-18T22:57:57.883 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T22:57:57.883 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-18T22:57:57.883 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-18T22:57:57.922 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:57:57.927 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:57:57.951 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-18T22:57:57.951 DEBUG:teuthology.orchestra.run.smithi142:> sudo update-grub 2024-04-18T22:57:57.969 INFO:teuthology.orchestra.run.smithi136.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:57:57.976 INFO:teuthology.orchestra.run.smithi136.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:57:58.094 INFO:teuthology.orchestra.run.smithi136.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:57:58.094 INFO:teuthology.orchestra.run.smithi136.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:57:58.094 INFO:teuthology.orchestra.run.smithi136.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:57:58.123 INFO:teuthology.orchestra.run.smithi136.stderr:done 2024-04-18T22:57:58.125 DEBUG:teuthology.orchestra.run.smithi136:> sudo shutdown -r now 2024-04-18T22:57:58.434 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-04-18T22:57:58.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:57:58.503 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-04-18T22:57:58.682 INFO:teuthology.orchestra.run.smithi073.stdout:Building dependency tree... 2024-04-18T22:57:58.682 INFO:teuthology.orchestra.run.smithi073.stdout:Reading state information... 2024-04-18T22:57:58.839 INFO:teuthology.orchestra.run.smithi073.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-18T22:57:58.840 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:57:58.840 INFO:teuthology.orchestra.run.smithi073.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:57:58.840 INFO:teuthology.orchestra.run.smithi073.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:57:58.840 INFO:teuthology.orchestra.run.smithi073.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:57:58.840 INFO:teuthology.orchestra.run.smithi073.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:57:58.840 INFO:teuthology.orchestra.run.smithi073.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:57:58.840 INFO:teuthology.orchestra.run.smithi073.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:57:58.878 INFO:teuthology.orchestra.run.smithi073.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:57:58.880 DEBUG:teuthology.orchestra.run.smithi073:> dpkg -s linux-image-generic 2024-04-18T22:57:58.933 INFO:teuthology.orchestra.run.smithi073.stdout:Package: linux-image-generic 2024-04-18T22:57:58.933 INFO:teuthology.orchestra.run.smithi073.stdout:Status: install ok installed 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Priority: optional 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Section: kernel 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Installed-Size: 21 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Architecture: amd64 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Source: linux-meta 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Version: 5.15.0.105.102 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Recommends: thermald 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout:Description: Generic Linux kernel image 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:57:58.934 INFO:teuthology.orchestra.run.smithi073.stdout: available. 2024-04-18T22:57:58.935 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-04-18T22:57:58.978 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.yOCMfgaw4p 2024-04-18T22:57:58.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo cp /boot/grub/grub.cfg /tmp/tmp.yOCMfgaw4p 2024-04-18T22:57:59.034 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.yOCMfgaw4p 2024-04-18T22:57:59.134 INFO:teuthology.orchestra.run.smithi142.stderr:Sourcing file `/etc/default/grub' 2024-04-18T22:57:59.135 INFO:teuthology.orchestra.run.smithi142.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:57:59.137 INFO:teuthology.orchestra.run.smithi142.stderr:Generating grub configuration file ... 2024-04-18T22:57:59.151 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.yOCMfgaw4p is 10KB 2024-04-18T22:57:59.164 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.yOCMfgaw4p 2024-04-18T22:57:59.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-18T22:57:59.223 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T22:57:59.223 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-18T22:57:59.223 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-18T22:57:59.289 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-18T22:57:59.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo update-grub 2024-04-18T22:57:59.345 INFO:teuthology.orchestra.run.smithi142.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:57:59.351 INFO:teuthology.orchestra.run.smithi142.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:57:59.643 INFO:teuthology.orchestra.run.smithi142.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:57:59.648 INFO:teuthology.orchestra.run.smithi142.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:57:59.691 INFO:teuthology.orchestra.run.smithi142.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:57:59.696 INFO:teuthology.orchestra.run.smithi142.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:57:59.822 INFO:teuthology.orchestra.run.smithi142.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:57:59.822 INFO:teuthology.orchestra.run.smithi142.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:57:59.822 INFO:teuthology.orchestra.run.smithi142.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:57:59.838 INFO:teuthology.orchestra.run.smithi142.stderr:done 2024-04-18T22:57:59.841 DEBUG:teuthology.orchestra.run.smithi142:> sudo shutdown -r now 2024-04-18T22:58:00.511 INFO:teuthology.orchestra.run.smithi073.stderr:Sourcing file `/etc/default/grub' 2024-04-18T22:58:00.512 INFO:teuthology.orchestra.run.smithi073.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:58:00.515 INFO:teuthology.orchestra.run.smithi073.stderr:Generating grub configuration file ... 2024-04-18T22:58:00.693 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:58:00.700 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:58:00.986 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:58:00.991 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:58:01.028 INFO:teuthology.orchestra.run.smithi073.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:58:01.033 INFO:teuthology.orchestra.run.smithi073.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:58:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:58:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:58:01.150 INFO:teuthology.orchestra.run.smithi073.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:58:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:done 2024-04-18T22:58:01.172 DEBUG:teuthology.orchestra.run.smithi073:> sudo shutdown -r now 2024-04-18T22:58:01.897 INFO:teuthology.orchestra.run.smithi120.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-04-18T22:58:01.897 INFO:teuthology.orchestra.run.smithi120.stdout:Sourcing file `/etc/default/grub' 2024-04-18T22:58:01.921 INFO:teuthology.orchestra.run.smithi120.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:58:01.943 INFO:teuthology.orchestra.run.smithi120.stdout:Generating grub configuration file ... 2024-04-18T22:58:02.192 INFO:teuthology.orchestra.run.smithi120.stdout:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:58:02.212 INFO:teuthology.orchestra.run.smithi120.stdout:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:58:02.553 INFO:teuthology.orchestra.run.smithi120.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:58:02.559 INFO:teuthology.orchestra.run.smithi120.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:58:02.627 INFO:teuthology.orchestra.run.smithi120.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:58:02.633 INFO:teuthology.orchestra.run.smithi120.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:58:02.822 INFO:teuthology.orchestra.run.smithi120.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:58:02.822 INFO:teuthology.orchestra.run.smithi120.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:58:02.822 INFO:teuthology.orchestra.run.smithi120.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:58:02.863 INFO:teuthology.orchestra.run.smithi120.stdout:done 2024-04-18T22:58:03.358 DEBUG:teuthology.orchestra.run.smithi120:> dpkg -s linux-image-generic 2024-04-18T22:58:03.388 INFO:teuthology.orchestra.run.smithi120.stdout:Package: linux-image-generic 2024-04-18T22:58:03.388 INFO:teuthology.orchestra.run.smithi120.stdout:Status: install ok installed 2024-04-18T22:58:03.388 INFO:teuthology.orchestra.run.smithi120.stdout:Priority: optional 2024-04-18T22:58:03.388 INFO:teuthology.orchestra.run.smithi120.stdout:Section: kernel 2024-04-18T22:58:03.388 INFO:teuthology.orchestra.run.smithi120.stdout:Installed-Size: 21 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: amd64 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout:Source: linux-meta 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout:Version: 5.15.0.105.102 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout:Recommends: thermald 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout:Description: Generic Linux kernel image 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:58:03.389 INFO:teuthology.orchestra.run.smithi120.stdout: available. 2024-04-18T22:58:03.389 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-105-generic 2024-04-18T22:58:03.390 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-04-18T22:58:03.390 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-04-18T22:58:03.390 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-04-18T22:58:03.390 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi120.front.sepia.ceph.com, path=None, version=distro) 2024-04-18T22:58:03.390 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get clean 2024-04-18T22:58:03.554 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get update 2024-04-18T22:58:03.680 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T22:58:03.680 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T22:58:03.685 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T22:58:03.705 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T22:58:05.129 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-04-18T22:58:05.146 DEBUG:teuthology.orchestra.run.smithi120:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-04-18T22:58:05.199 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-04-18T22:58:05.376 INFO:teuthology.orchestra.run.smithi120.stdout:Building dependency tree... 2024-04-18T22:58:05.376 INFO:teuthology.orchestra.run.smithi120.stdout:Reading state information... 2024-04-18T22:58:05.533 INFO:teuthology.orchestra.run.smithi120.stdout:linux-image-generic is already the newest version (5.15.0.105.102). 2024-04-18T22:58:05.533 INFO:teuthology.orchestra.run.smithi120.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T22:58:05.534 INFO:teuthology.orchestra.run.smithi120.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T22:58:05.534 INFO:teuthology.orchestra.run.smithi120.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T22:58:05.534 INFO:teuthology.orchestra.run.smithi120.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T22:58:05.534 INFO:teuthology.orchestra.run.smithi120.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T22:58:05.534 INFO:teuthology.orchestra.run.smithi120.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T22:58:05.534 INFO:teuthology.orchestra.run.smithi120.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T22:58:05.574 INFO:teuthology.orchestra.run.smithi120.stdout:0 upgraded, 0 newly installed, 0 to remove and 315 not upgraded. 2024-04-18T22:58:05.576 DEBUG:teuthology.orchestra.run.smithi120:> dpkg -s linux-image-generic 2024-04-18T22:58:05.633 INFO:teuthology.orchestra.run.smithi120.stdout:Package: linux-image-generic 2024-04-18T22:58:05.633 INFO:teuthology.orchestra.run.smithi120.stdout:Status: install ok installed 2024-04-18T22:58:05.633 INFO:teuthology.orchestra.run.smithi120.stdout:Priority: optional 2024-04-18T22:58:05.633 INFO:teuthology.orchestra.run.smithi120.stdout:Section: kernel 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout:Installed-Size: 21 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout:Maintainer: Ubuntu Kernel Team 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout:Architecture: amd64 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout:Source: linux-meta 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout:Version: 5.15.0.105.102 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.3), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-105), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.3) 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout:Depends: linux-image-5.15.0-105-generic, linux-modules-extra-5.15.0-105-generic, linux-firmware, intel-microcode, amd64-microcode 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout:Recommends: thermald 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout:Description: Generic Linux kernel image 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout: This package will always depend on the latest generic kernel image 2024-04-18T22:58:05.634 INFO:teuthology.orchestra.run.smithi120.stdout: available. 2024-04-18T22:58:05.635 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-04-18T22:58:05.678 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.nSfWazHTyQ 2024-04-18T22:58:05.678 DEBUG:teuthology.orchestra.run.smithi120:> sudo cp /boot/grub/grub.cfg /tmp/tmp.nSfWazHTyQ 2024-04-18T22:58:05.750 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.nSfWazHTyQ 2024-04-18T22:58:05.882 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.nSfWazHTyQ is 10KB 2024-04-18T22:58:05.895 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.nSfWazHTyQ 2024-04-18T22:58:05.901 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-04-18T22:58:05.954 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T22:58:05.955 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-04-18T22:58:05.955 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-04-18T22:58:06.039 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-105-generic 2024-04-18T22:58:06.039 DEBUG:teuthology.orchestra.run.smithi120:> sudo update-grub 2024-04-18T22:58:07.369 INFO:teuthology.orchestra.run.smithi120.stderr:Sourcing file `/etc/default/grub' 2024-04-18T22:58:07.370 INFO:teuthology.orchestra.run.smithi120.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-04-18T22:58:07.372 INFO:teuthology.orchestra.run.smithi120.stderr:Generating grub configuration file ... 2024-04-18T22:58:07.599 INFO:teuthology.orchestra.run.smithi120.stderr:Found linux image: /boot/vmlinuz-5.15.0-105-generic 2024-04-18T22:58:07.607 INFO:teuthology.orchestra.run.smithi120.stderr:Found initrd image: /boot/initrd.img-5.15.0-105-generic 2024-04-18T22:58:07.868 INFO:teuthology.orchestra.run.smithi120.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-04-18T22:58:07.873 INFO:teuthology.orchestra.run.smithi120.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-04-18T22:58:07.911 INFO:teuthology.orchestra.run.smithi120.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-04-18T22:58:07.915 INFO:teuthology.orchestra.run.smithi120.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-04-18T22:58:08.020 INFO:teuthology.orchestra.run.smithi120.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-04-18T22:58:08.020 INFO:teuthology.orchestra.run.smithi120.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-04-18T22:58:08.020 INFO:teuthology.orchestra.run.smithi120.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-04-18T22:58:08.034 INFO:teuthology.orchestra.run.smithi120.stderr:done 2024-04-18T22:58:08.036 DEBUG:teuthology.orchestra.run.smithi120:> sudo shutdown -r now 2024-04-18T22:58:28.190 INFO:teuthology.task.kernel:Checking client mon.c for new kernel version... 2024-04-18T22:58:28.191 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-18T22:58:28.192 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:58:29.843 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2024-04-18T22:58:29.844 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-18T22:58:29.844 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:58:31.176 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-04-18T22:58:31.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-18T22:58:31.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:58:38.042 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-04-18T22:58:38.042 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-04-18T22:58:38.043 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:59:02.819 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-04-18T22:59:05.634 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-04-18T22:59:11.822 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-18T22:59:11.823 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:59:14.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-18T22:59:14.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:59:29.862 DEBUG:teuthology.orchestra.remote:timed out 2024-04-18T22:59:30.206 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.136 2024-04-18T22:59:33.026 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.73 2024-04-18T22:59:38.046 DEBUG:teuthology.orchestra.remote:timed out 2024-04-18T22:59:38.864 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-18T22:59:38.864 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:59:38.867 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.142 2024-04-18T22:59:42.210 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-18T22:59:42.211 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:59:42.576 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-04-18T22:59:43.161 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-18T22:59:43.161 INFO:teuthology.task.kernel:Checking kernel version of mon.c, want "5.15.0-105-generic"... 2024-04-18T22:59:43.161 DEBUG:teuthology.orchestra.run.smithi136:> uname -r 2024-04-18T22:59:43.208 INFO:teuthology.orchestra.run.smithi136.stdout:5.15.0-105-generic 2024-04-18T22:59:43.209 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-18T22:59:43.209 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-18T22:59:43.209 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-18T22:59:44.209 INFO:teuthology.task.kernel:Enabling kdb on mon.c... 2024-04-18T22:59:44.210 DEBUG:teuthology.orchestra.run.smithi136:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-18T22:59:44.286 INFO:teuthology.orchestra.run.smithi136.stdout:ttyS1 2024-04-18T22:59:44.314 DEBUG:teuthology.parallel:result is None 2024-04-18T22:59:45.027 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-18T22:59:45.028 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:59:45.646 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-04-18T22:59:46.447 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-18T22:59:46.447 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-105-generic"... 2024-04-18T22:59:46.447 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2024-04-18T22:59:46.453 INFO:teuthology.orchestra.run.smithi073.stdout:5.15.0-105-generic 2024-04-18T22:59:46.453 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-18T22:59:46.453 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-18T22:59:46.453 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-18T22:59:47.047 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-04-18T22:59:47.048 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:59:47.454 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-18T22:59:47.454 DEBUG:teuthology.orchestra.run.smithi073:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-18T22:59:47.625 INFO:teuthology.orchestra.run.smithi073.stdout:ttyS1 2024-04-18T22:59:47.656 DEBUG:teuthology.parallel:result is None 2024-04-18T22:59:50.733 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-04-18T22:59:50.868 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-18T22:59:50.869 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T22:59:51.079 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-04-18T22:59:51.635 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-18T22:59:51.635 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.15.0-105-generic"... 2024-04-18T22:59:51.635 DEBUG:teuthology.orchestra.run.smithi142:> uname -r 2024-04-18T22:59:51.684 INFO:teuthology.orchestra.run.smithi142.stdout:5.15.0-105-generic 2024-04-18T22:59:51.685 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-18T22:59:51.685 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-18T22:59:51.685 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-18T22:59:51.769 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-04-18T22:59:51.769 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-105-generic"... 2024-04-18T22:59:51.769 DEBUG:teuthology.orchestra.run.smithi120:> uname -r 2024-04-18T22:59:51.816 INFO:teuthology.orchestra.run.smithi120.stdout:5.15.0-105-generic 2024-04-18T22:59:51.816 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-105-generic vs 5.15.0-105-generic 2024-04-18T22:59:51.816 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-04-18T22:59:51.816 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-04-18T22:59:52.686 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-18T22:59:52.686 DEBUG:teuthology.orchestra.run.smithi142:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-18T22:59:52.817 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-18T22:59:52.818 DEBUG:teuthology.orchestra.run.smithi120:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-18T22:59:52.992 INFO:teuthology.orchestra.run.smithi120.stdout:ttyS1 2024-04-18T22:59:53.011 DEBUG:teuthology.parallel:result is None 2024-04-18T22:59:53.050 INFO:teuthology.orchestra.run.smithi142.stdout:ttyS1 2024-04-18T22:59:53.081 DEBUG:teuthology.parallel:result is None 2024-04-18T22:59:53.082 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-18T22:59:53.091 INFO:teuthology.task.internal:Creating test directory... 2024-04-18T22:59:53.091 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-18T22:59:53.094 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-18T22:59:53.096 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-18T22:59:53.100 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-18T22:59:53.106 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-18T22:59:53.113 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-18T22:59:53.120 INFO:teuthology.task.internal:Creating archive directory... 2024-04-18T22:59:53.120 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-18T22:59:53.144 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-18T22:59:53.146 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-18T22:59:53.148 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-18T22:59:53.226 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-18T22:59:53.235 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-18T22:59:53.235 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-18T22:59:53.239 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-18T22:59:53.241 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-18T22:59:53.243 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-18T22:59:53.255 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:59:53.263 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:59:53.264 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:59:53.264 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:59:53.271 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:59:53.272 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:59:53.272 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:59:53.281 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-18T22:59:53.282 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-18T22:59:53.290 INFO:teuthology.task.internal:Configuring sudo... 2024-04-18T22:59:53.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-18T22:59:53.312 DEBUG:teuthology.orchestra.run.smithi120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-18T22:59:53.325 DEBUG:teuthology.orchestra.run.smithi136:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-18T22:59:53.327 DEBUG:teuthology.orchestra.run.smithi142:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-18T22:59:53.348 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-18T22:59:53.357 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-18T22:59:53.357 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-18T22:59:53.368 DEBUG:teuthology.orchestra.run.smithi120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-18T22:59:53.385 DEBUG:teuthology.orchestra.run.smithi136:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-18T22:59:53.389 DEBUG:teuthology.orchestra.run.smithi142:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-18T22:59:53.396 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-18T22:59:53.419 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-18T22:59:53.469 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T22:59:53.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-18T22:59:53.535 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-18T22:59:53.540 DEBUG:teuthology.orchestra.run.smithi120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-18T22:59:53.589 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T22:59:53.589 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-18T22:59:53.672 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-18T22:59:53.679 DEBUG:teuthology.orchestra.run.smithi136:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-18T22:59:53.728 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T22:59:53.728 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-18T22:59:53.801 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-18T22:59:53.807 DEBUG:teuthology.orchestra.run.smithi142:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-18T22:59:53.853 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T22:59:53.853 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-18T22:59:53.940 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2024-04-18T22:59:53.942 DEBUG:teuthology.orchestra.run.smithi120:> sudo service rsyslog restart 2024-04-18T22:59:53.945 DEBUG:teuthology.orchestra.run.smithi136:> sudo service rsyslog restart 2024-04-18T22:59:53.948 DEBUG:teuthology.orchestra.run.smithi142:> sudo service rsyslog restart 2024-04-18T22:59:54.046 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-18T22:59:54.055 INFO:teuthology.task.internal:Starting timer... 2024-04-18T22:59:54.055 INFO:teuthology.run_tasks:Running task pcp... 2024-04-18T22:59:54.072 INFO:teuthology.run_tasks:Running task selinux... 2024-04-18T22:59:54.081 DEBUG:teuthology.task.selinux:Excluding smithi073: OS 'ubuntu' does not support SELinux 2024-04-18T22:59:54.081 DEBUG:teuthology.task.selinux:Excluding smithi120: OS 'ubuntu' does not support SELinux 2024-04-18T22:59:54.081 DEBUG:teuthology.task.selinux:Excluding smithi136: OS 'ubuntu' does not support SELinux 2024-04-18T22:59:54.081 DEBUG:teuthology.task.selinux:Excluding smithi142: OS 'ubuntu' does not support SELinux 2024-04-18T22:59:54.081 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-18T22:59:54.082 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-04-18T22:59:54.082 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-18T22:59:54.082 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-18T22:59:54.091 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-18T22:59:54.098 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-18T22:59:54.225 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-18T22:59:54.245 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-18T22:59:54.247 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi073.front.sepia.ceph.com,smithi120.front.sepia.ceph.com,smithi136.front.sepia.ceph.com,smithi142.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-18T23:07:06.317 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi120.front.sepia.ceph.com'), Remote(name='ubuntu@smithi136.front.sepia.ceph.com'), Remote(name='ubuntu@smithi142.front.sepia.ceph.com')] 2024-04-18T23:07:06.319 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-18T23:07:06.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T23:07:06.403 DEBUG:teuthology.orchestra.run.smithi073:> true 2024-04-18T23:07:06.482 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2024-04-18T23:07:06.482 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-04-18T23:07:06.483 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T23:07:06.563 DEBUG:teuthology.orchestra.run.smithi120:> true 2024-04-18T23:07:06.642 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi120.front.sepia.ceph.com' 2024-04-18T23:07:06.642 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-18T23:07:06.643 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi136.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T23:07:06.722 DEBUG:teuthology.orchestra.run.smithi136:> true 2024-04-18T23:07:06.798 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi136.front.sepia.ceph.com' 2024-04-18T23:07:06.798 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-18T23:07:06.799 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi142.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-18T23:07:06.877 DEBUG:teuthology.orchestra.run.smithi142:> true 2024-04-18T23:07:06.950 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi142.front.sepia.ceph.com' 2024-04-18T23:07:06.951 INFO:teuthology.run_tasks:Running task clock... 2024-04-18T23:07:06.963 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-18T23:07:06.963 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-18T23:07:06.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T23:07:06.966 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-18T23:07:06.966 DEBUG:teuthology.orchestra.run.smithi120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T23:07:06.969 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-18T23:07:06.969 DEBUG:teuthology.orchestra.run.smithi136:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T23:07:06.971 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-18T23:07:06.971 DEBUG:teuthology.orchestra.run.smithi142:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T23:07:06.992 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-18T23:07:06.992 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: Command line: ntpd -gq 2024-04-18T23:07:06.992 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: ---------------------------------------------------- 2024-04-18T23:07:06.992 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: ntp-4 is maintained by Network Time Foundation, 2024-04-18T23:07:06.992 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-18T23:07:06.992 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: corporation. Support and training for ntp-4 are 2024-04-18T23:07:06.993 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: available at https://www.nwtime.org/support 2024-04-18T23:07:06.993 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: ---------------------------------------------------- 2024-04-18T23:07:06.993 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: proto: precision = 0.045 usec (-24) 2024-04-18T23:07:06.993 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: basedate set to 2022-02-04 2024-04-18T23:07:06.993 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: gps base set to 2022-02-06 (week 2196) 2024-04-18T23:07:06.994 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T23:07:06.994 INFO:teuthology.orchestra.run.smithi073.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T23:07:06.994 INFO:teuthology.orchestra.run.smithi073.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-18T23:07:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: restrict ::: KOD does nothing without LIMITED. 2024-04-18T23:07:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: Listen and drop on 0 v6wildcard [::]:123 2024-04-18T23:07:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-18T23:07:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: Listen normally on 2 lo 127.0.0.1:123 2024-04-18T23:07:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: Listen normally on 3 enp3s0f1 172.21.15.73:123 2024-04-18T23:07:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: Listen normally on 4 lo [::1]:123 2024-04-18T23:07:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:ce7d%5]:123 2024-04-18T23:07:06.995 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:06 ntpd[17203]: Listening on routing socket on fd #22 for interface updates 2024-04-18T23:07:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-18T23:07:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: Command line: ntpd -gq 2024-04-18T23:07:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: ---------------------------------------------------- 2024-04-18T23:07:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: ntp-4 is maintained by Network Time Foundation, 2024-04-18T23:07:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-18T23:07:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: corporation. Support and training for ntp-4 are 2024-04-18T23:07:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: available at https://www.nwtime.org/support 2024-04-18T23:07:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: ---------------------------------------------------- 2024-04-18T23:07:06.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: proto: precision = 0.060 usec (-24) 2024-04-18T23:07:06.998 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: basedate set to 2022-02-04 2024-04-18T23:07:06.998 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: gps base set to 2022-02-06 (week 2196) 2024-04-18T23:07:06.998 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T23:07:06.998 INFO:teuthology.orchestra.run.smithi120.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T23:07:06.998 INFO:teuthology.orchestra.run.smithi120.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-18T23:07:06.999 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: restrict ::: KOD does nothing without LIMITED. 2024-04-18T23:07:06.999 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: Listen and drop on 0 v6wildcard [::]:123 2024-04-18T23:07:06.999 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-18T23:07:06.999 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: Listen normally on 2 lo 127.0.0.1:123 2024-04-18T23:07:06.999 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: Listen normally on 3 enp3s0f1 172.21.15.120:123 2024-04-18T23:07:06.999 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: Listen normally on 4 lo [::1]:123 2024-04-18T23:07:07.000 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:478f%5]:123 2024-04-18T23:07:07.000 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:06 ntpd[17207]: Listening on routing socket on fd #22 for interface updates 2024-04-18T23:07:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-18T23:07:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: Command line: ntpd -gq 2024-04-18T23:07:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: ---------------------------------------------------- 2024-04-18T23:07:07.001 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: ntp-4 is maintained by Network Time Foundation, 2024-04-18T23:07:07.002 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-18T23:07:07.002 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: corporation. Support and training for ntp-4 are 2024-04-18T23:07:07.002 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: available at https://www.nwtime.org/support 2024-04-18T23:07:07.002 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: ---------------------------------------------------- 2024-04-18T23:07:07.002 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: proto: precision = 0.045 usec (-24) 2024-04-18T23:07:07.002 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: basedate set to 2022-02-04 2024-04-18T23:07:07.002 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: gps base set to 2022-02-06 (week 2196) 2024-04-18T23:07:07.002 INFO:teuthology.orchestra.run.smithi136.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T23:07:07.002 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T23:07:07.003 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:06 ntpd[17187]: restrict ::: KOD does nothing without LIMITED. 2024-04-18T23:07:07.003 INFO:teuthology.orchestra.run.smithi136.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-18T23:07:07.004 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:07 ntpd[17187]: Listen and drop on 0 v6wildcard [::]:123 2024-04-18T23:07:07.004 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:07 ntpd[17187]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-18T23:07:07.004 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:07 ntpd[17187]: Listen normally on 2 lo 127.0.0.1:123 2024-04-18T23:07:07.004 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:07 ntpd[17187]: Listen normally on 3 enp3s0f1 172.21.15.136:123 2024-04-18T23:07:07.004 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:07 ntpd[17187]: Listen normally on 4 lo [::1]:123 2024-04-18T23:07:07.004 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:07 ntpd[17187]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e43%5]:123 2024-04-18T23:07:07.004 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:07 ntpd[17187]: Listening on routing socket on fd #22 for interface updates 2024-04-18T23:07:07.020 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-04-18T23:07:07.020 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: Command line: ntpd -gq 2024-04-18T23:07:07.020 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: ---------------------------------------------------- 2024-04-18T23:07:07.020 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: ntp-4 is maintained by Network Time Foundation, 2024-04-18T23:07:07.020 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-04-18T23:07:07.020 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: corporation. Support and training for ntp-4 are 2024-04-18T23:07:07.020 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: available at https://www.nwtime.org/support 2024-04-18T23:07:07.020 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: ---------------------------------------------------- 2024-04-18T23:07:07.021 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: proto: precision = 0.045 usec (-24) 2024-04-18T23:07:07.021 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: basedate set to 2022-02-04 2024-04-18T23:07:07.021 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: gps base set to 2022-02-06 (week 2196) 2024-04-18T23:07:07.021 INFO:teuthology.orchestra.run.smithi142.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T23:07:07.021 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-04-18T23:07:07.021 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: restrict ::: KOD does nothing without LIMITED. 2024-04-18T23:07:07.022 INFO:teuthology.orchestra.run.smithi142.stderr:restrict ::: KOD does nothing without LIMITED. 2024-04-18T23:07:07.023 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: Listen and drop on 0 v6wildcard [::]:123 2024-04-18T23:07:07.023 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-04-18T23:07:07.023 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: Listen normally on 2 lo 127.0.0.1:123 2024-04-18T23:07:07.023 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: Listen normally on 3 enp3s0f1 172.21.15.142:123 2024-04-18T23:07:07.023 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: Listen normally on 4 lo [::1]:123 2024-04-18T23:07:07.023 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6e0d%5]:123 2024-04-18T23:07:07.023 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:07 ntpd[17186]: Listening on routing socket on fd #22 for interface updates 2024-04-18T23:07:07.996 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:07 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:07.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:07 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:07.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:08 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:07.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:08 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:07.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:08 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:07.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:08 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:07.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:08 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:07.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:08 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:08.020 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:08 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:08.020 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:08 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:08.993 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:08 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:08.993 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:08 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:08.993 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:08 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:08.993 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:08 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:08.993 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:08 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:08.993 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:08 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:08.993 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:08 ntpd[17203]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:08.993 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:08 ntpd[17203]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:08.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:08 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:08.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:08 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:08.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:08 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:08.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:08 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:08.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:08 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:08.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:08 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:09.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:09 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:09.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:09 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:09.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:09 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:09.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:09 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:09.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:09 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:09.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:09 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:10.020 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:10.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:10 ntpd[17186]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:10.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:10.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:10 ntpd[17203]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:10.996 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:10 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:10 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:10 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:10 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:10 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:10.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:10 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:10.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:11 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:11 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:11 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:10.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:11 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:10.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:11 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:10.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:11 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:11.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:11 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.000 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.000 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.000 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.000 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.001 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:12 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:12 ntpd[17186]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:12 ntpd[17203]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.996 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:12 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:12 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:12 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:12 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:12 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:12 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:12.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:13 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:13 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:13 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:12.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:13 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:12.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:13 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:12.998 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:13 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:13.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:13.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:13.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:13.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:13.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:13.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:13.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:13.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:13.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:13.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:13.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:13.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:13 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:13.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:13.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:13.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:13.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:13.999 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.000 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.000 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:14.000 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:14.000 INFO:teuthology.orchestra.run.smithi136.stdout:18 Apr 23:07:14 ntpd[17187]: ntpd: time slew -0.001101 s 2024-04-18T23:07:14.000 INFO:teuthology.orchestra.run.smithi136.stdout:ntpd: time slew -0.001101s 2024-04-18T23:07:14.000 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-18T23:07:14.000 INFO:teuthology.orchestra.run.smithi136.stderr:18 Apr 23:07:14 ntpd[17187]: can't open /var/log/ntpstats/loopstats.20240418: Permission denied 2024-04-18T23:07:14.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:14.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:14 ntpd[17186]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:14.045 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T23:07:14.046 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-04-18T23:07:14.046 INFO:teuthology.orchestra.run.smithi136.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:14.046 INFO:teuthology.orchestra.run.smithi136.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:14.046 INFO:teuthology.orchestra.run.smithi136.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:14.046 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:14.994 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:14.995 INFO:teuthology.orchestra.run.smithi073.stdout:18 Apr 23:07:14 ntpd[17203]: ntpd: time slew +0.001619 s 2024-04-18T23:07:14.995 INFO:teuthology.orchestra.run.smithi073.stdout:ntpd: time slew +0.001619s 2024-04-18T23:07:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-18T23:07:14.995 INFO:teuthology.orchestra.run.smithi073.stderr:18 Apr 23:07:14 ntpd[17203]: can't open /var/log/ntpstats/loopstats.20240418: Permission denied 2024-04-18T23:07:14.996 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:14 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:14 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:14 ntpd[17207]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:14 ntpd[17207]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:14 ntpd[17207]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:14.997 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:14 ntpd[17207]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:14.997 INFO:teuthology.orchestra.run.smithi120.stdout:18 Apr 23:07:14 ntpd[17207]: ntpd: time slew -0.001099 s 2024-04-18T23:07:14.997 INFO:teuthology.orchestra.run.smithi120.stdout:ntpd: time slew -0.001099s 2024-04-18T23:07:14.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:14 ntpd[17207]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-18T23:07:14.998 INFO:teuthology.orchestra.run.smithi120.stderr:18 Apr 23:07:14 ntpd[17207]: can't open /var/log/ntpstats/loopstats.20240418: Permission denied 2024-04-18T23:07:15.044 INFO:teuthology.orchestra.run.smithi120.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T23:07:15.044 INFO:teuthology.orchestra.run.smithi120.stdout:============================================================================== 2024-04-18T23:07:15.044 INFO:teuthology.orchestra.run.smithi120.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:15.044 INFO:teuthology.orchestra.run.smithi120.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:15.044 INFO:teuthology.orchestra.run.smithi120.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:15.044 INFO:teuthology.orchestra.run.smithi120.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:15.049 INFO:teuthology.orchestra.run.smithi073.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T23:07:15.049 INFO:teuthology.orchestra.run.smithi073.stdout:============================================================================== 2024-04-18T23:07:15.050 INFO:teuthology.orchestra.run.smithi073.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:15.050 INFO:teuthology.orchestra.run.smithi073.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:15.050 INFO:teuthology.orchestra.run.smithi073.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:15.050 INFO:teuthology.orchestra.run.smithi073.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:16.020 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: can't open /var/log/ntpstats/rawstats.20240418: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: can't open /var/log/ntpstats/peerstats.20240418: Permission denied 2024-04-18T23:07:16.021 INFO:teuthology.orchestra.run.smithi142.stdout:18 Apr 23:07:16 ntpd[17186]: ntpd: time slew -0.000161 s 2024-04-18T23:07:16.022 INFO:teuthology.orchestra.run.smithi142.stdout:ntpd: time slew -0.000161s 2024-04-18T23:07:16.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-04-18T23:07:16.022 INFO:teuthology.orchestra.run.smithi142.stderr:18 Apr 23:07:16 ntpd[17186]: can't open /var/log/ntpstats/loopstats.20240418: Permission denied 2024-04-18T23:07:16.074 INFO:teuthology.orchestra.run.smithi142.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T23:07:16.074 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================== 2024-04-18T23:07:16.074 INFO:teuthology.orchestra.run.smithi142.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:16.074 INFO:teuthology.orchestra.run.smithi142.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:16.074 INFO:teuthology.orchestra.run.smithi142.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:16.074 INFO:teuthology.orchestra.run.smithi142.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:07:16.075 INFO:teuthology.run_tasks:Running task install... 2024-04-18T23:07:16.087 DEBUG:teuthology.task.install:project ceph 2024-04-18T23:07:16.087 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}} 2024-04-18T23:07:16.087 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'} 2024-04-18T23:07:16.088 INFO:teuthology.task.install:Using flavor: default 2024-04-18T23:07:16.093 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-18T23:07:16.093 INFO:teuthology.task.install:extra packages: [] 2024-04-18T23:07:16.093 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-key list | grep Ceph 2024-04-18T23:07:16.094 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-key list | grep Ceph 2024-04-18T23:07:16.094 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-key list | grep Ceph 2024-04-18T23:07:16.095 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-key list | grep Ceph 2024-04-18T23:07:16.160 INFO:teuthology.orchestra.run.smithi136.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-18T23:07:16.161 INFO:teuthology.orchestra.run.smithi073.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-18T23:07:16.162 INFO:teuthology.orchestra.run.smithi120.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-18T23:07:16.174 INFO:teuthology.orchestra.run.smithi142.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-04-18T23:07:16.190 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-18T23:07:16.190 INFO:teuthology.orchestra.run.smithi136.stdout:uid [ unknown] Ceph.com (release key) 2024-04-18T23:07:16.190 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-18T23:07:16.191 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T23:07:16.196 INFO:teuthology.orchestra.run.smithi120.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-18T23:07:16.196 INFO:teuthology.orchestra.run.smithi120.stdout:uid [ unknown] Ceph.com (release key) 2024-04-18T23:07:16.197 INFO:teuthology.orchestra.run.smithi073.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-18T23:07:16.197 INFO:teuthology.orchestra.run.smithi073.stdout:uid [ unknown] Ceph.com (release key) 2024-04-18T23:07:16.197 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-18T23:07:16.197 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T23:07:16.198 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-18T23:07:16.198 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T23:07:16.205 INFO:teuthology.orchestra.run.smithi142.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-04-18T23:07:16.205 INFO:teuthology.orchestra.run.smithi142.stdout:uid [ unknown] Ceph.com (release key) 2024-04-18T23:07:16.205 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-04-18T23:07:16.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T23:07:16.330 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default/ 2024-04-18T23:07:16.330 INFO:teuthology.task.install.deb:Package version is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:07:16.346 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default/ 2024-04-18T23:07:16.346 INFO:teuthology.task.install.deb:Package version is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:07:16.415 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default/ 2024-04-18T23:07:16.415 INFO:teuthology.task.install.deb:Package version is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:07:16.447 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default/ 2024-04-18T23:07:16.447 INFO:teuthology.task.install.deb:Package version is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:07:16.460 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:07:16.460 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-18T23:07:16.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo apt-get update 2024-04-18T23:07:16.495 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:07:16.495 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-18T23:07:16.508 DEBUG:teuthology.orchestra.run.smithi136:> sudo apt-get update 2024-04-18T23:07:16.520 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:07:16.520 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-18T23:07:16.534 DEBUG:teuthology.orchestra.run.smithi073:> sudo apt-get update 2024-04-18T23:07:16.551 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:07:16.551 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-04-18T23:07:16.565 DEBUG:teuthology.orchestra.run.smithi142:> sudo apt-get update 2024-04-18T23:07:16.679 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T23:07:16.680 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T23:07:16.708 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T23:07:16.718 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T23:07:16.729 INFO:teuthology.orchestra.run.smithi136.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T23:07:16.758 INFO:teuthology.orchestra.run.smithi120.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy InRelease 2024-04-18T23:07:16.786 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy InRelease 2024-04-18T23:07:16.795 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T23:07:16.796 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T23:07:16.796 INFO:teuthology.orchestra.run.smithi120.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-18T23:07:16.814 INFO:teuthology.orchestra.run.smithi073.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy InRelease 2024-04-18T23:07:16.822 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T23:07:16.822 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-18T23:07:16.833 INFO:teuthology.orchestra.run.smithi120.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-18T23:07:16.846 INFO:teuthology.orchestra.run.smithi142.stdout:Ign:1 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy InRelease 2024-04-18T23:07:16.851 INFO:teuthology.orchestra.run.smithi073.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-18T23:07:16.859 INFO:teuthology.orchestra.run.smithi136.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-18T23:07:16.860 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-04-18T23:07:16.861 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-04-18T23:07:16.868 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T23:07:16.869 INFO:teuthology.orchestra.run.smithi120.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-18T23:07:16.883 INFO:teuthology.orchestra.run.smithi142.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-04-18T23:07:16.888 INFO:teuthology.orchestra.run.smithi073.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-18T23:07:16.897 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-18T23:07:16.906 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T23:07:16.915 INFO:teuthology.orchestra.run.smithi120.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,425 B] 2024-04-18T23:07:16.920 INFO:teuthology.orchestra.run.smithi142.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy Release.gpg 2024-04-18T23:07:16.924 INFO:teuthology.orchestra.run.smithi073.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,425 B] 2024-04-18T23:07:16.932 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-04-18T23:07:16.937 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,425 B] 2024-04-18T23:07:16.950 INFO:teuthology.orchestra.run.smithi120.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T23:07:16.956 INFO:teuthology.orchestra.run.smithi142.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,425 B] 2024-04-18T23:07:16.967 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-18T23:07:16.985 INFO:teuthology.orchestra.run.smithi073.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T23:07:16.999 INFO:teuthology.orchestra.run.smithi142.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.9 kB] 2024-04-18T23:07:17.013 INFO:teuthology.orchestra.run.smithi142.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-04-18T23:07:17.015 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 29.0 kB in 0s (86.8 kB/s) 2024-04-18T23:07:17.123 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 29.0 kB in 0s (70.7 kB/s) 2024-04-18T23:07:17.179 INFO:teuthology.orchestra.run.smithi120.stdout:Fetched 29.0 kB in 1s (55.5 kB/s) 2024-04-18T23:07:17.225 INFO:teuthology.orchestra.run.smithi142.stdout:Fetched 29.0 kB in 0s (59.9 kB/s) 2024-04-18T23:07:17.966 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-04-18T23:07:17.981 DEBUG:teuthology.orchestra.run.smithi136:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2369-g1a6a3e74-1jammy cephadm=19.0.0-2369-g1a6a3e74-1jammy ceph-mds=19.0.0-2369-g1a6a3e74-1jammy ceph-mgr=19.0.0-2369-g1a6a3e74-1jammy ceph-common=19.0.0-2369-g1a6a3e74-1jammy ceph-fuse=19.0.0-2369-g1a6a3e74-1jammy ceph-test=19.0.0-2369-g1a6a3e74-1jammy ceph-volume=19.0.0-2369-g1a6a3e74-1jammy radosgw=19.0.0-2369-g1a6a3e74-1jammy python3-rados=19.0.0-2369-g1a6a3e74-1jammy python3-rgw=19.0.0-2369-g1a6a3e74-1jammy python3-cephfs=19.0.0-2369-g1a6a3e74-1jammy python3-rbd=19.0.0-2369-g1a6a3e74-1jammy libcephfs2=19.0.0-2369-g1a6a3e74-1jammy libcephfs-dev=19.0.0-2369-g1a6a3e74-1jammy librados2=19.0.0-2369-g1a6a3e74-1jammy librbd1=19.0.0-2369-g1a6a3e74-1jammy rbd-fuse=19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:07:18.043 INFO:teuthology.orchestra.run.smithi136.stdout:Reading package lists... 2024-04-18T23:07:18.045 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-04-18T23:07:18.061 DEBUG:teuthology.orchestra.run.smithi073:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2369-g1a6a3e74-1jammy cephadm=19.0.0-2369-g1a6a3e74-1jammy ceph-mds=19.0.0-2369-g1a6a3e74-1jammy ceph-mgr=19.0.0-2369-g1a6a3e74-1jammy ceph-common=19.0.0-2369-g1a6a3e74-1jammy ceph-fuse=19.0.0-2369-g1a6a3e74-1jammy ceph-test=19.0.0-2369-g1a6a3e74-1jammy ceph-volume=19.0.0-2369-g1a6a3e74-1jammy radosgw=19.0.0-2369-g1a6a3e74-1jammy python3-rados=19.0.0-2369-g1a6a3e74-1jammy python3-rgw=19.0.0-2369-g1a6a3e74-1jammy python3-cephfs=19.0.0-2369-g1a6a3e74-1jammy python3-rbd=19.0.0-2369-g1a6a3e74-1jammy libcephfs2=19.0.0-2369-g1a6a3e74-1jammy libcephfs-dev=19.0.0-2369-g1a6a3e74-1jammy librados2=19.0.0-2369-g1a6a3e74-1jammy librbd1=19.0.0-2369-g1a6a3e74-1jammy rbd-fuse=19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:07:18.121 INFO:teuthology.orchestra.run.smithi073.stdout:Reading package lists... 2024-04-18T23:07:18.154 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-04-18T23:07:18.171 DEBUG:teuthology.orchestra.run.smithi120:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2369-g1a6a3e74-1jammy cephadm=19.0.0-2369-g1a6a3e74-1jammy ceph-mds=19.0.0-2369-g1a6a3e74-1jammy ceph-mgr=19.0.0-2369-g1a6a3e74-1jammy ceph-common=19.0.0-2369-g1a6a3e74-1jammy ceph-fuse=19.0.0-2369-g1a6a3e74-1jammy ceph-test=19.0.0-2369-g1a6a3e74-1jammy ceph-volume=19.0.0-2369-g1a6a3e74-1jammy radosgw=19.0.0-2369-g1a6a3e74-1jammy python3-rados=19.0.0-2369-g1a6a3e74-1jammy python3-rgw=19.0.0-2369-g1a6a3e74-1jammy python3-cephfs=19.0.0-2369-g1a6a3e74-1jammy python3-rbd=19.0.0-2369-g1a6a3e74-1jammy libcephfs2=19.0.0-2369-g1a6a3e74-1jammy libcephfs-dev=19.0.0-2369-g1a6a3e74-1jammy librados2=19.0.0-2369-g1a6a3e74-1jammy librbd1=19.0.0-2369-g1a6a3e74-1jammy rbd-fuse=19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:07:18.215 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-04-18T23:07:18.229 INFO:teuthology.orchestra.run.smithi136.stdout:Building dependency tree... 2024-04-18T23:07:18.230 INFO:teuthology.orchestra.run.smithi136.stdout:Reading state information... 2024-04-18T23:07:18.231 DEBUG:teuthology.orchestra.run.smithi142:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-2369-g1a6a3e74-1jammy cephadm=19.0.0-2369-g1a6a3e74-1jammy ceph-mds=19.0.0-2369-g1a6a3e74-1jammy ceph-mgr=19.0.0-2369-g1a6a3e74-1jammy ceph-common=19.0.0-2369-g1a6a3e74-1jammy ceph-fuse=19.0.0-2369-g1a6a3e74-1jammy ceph-test=19.0.0-2369-g1a6a3e74-1jammy ceph-volume=19.0.0-2369-g1a6a3e74-1jammy radosgw=19.0.0-2369-g1a6a3e74-1jammy python3-rados=19.0.0-2369-g1a6a3e74-1jammy python3-rgw=19.0.0-2369-g1a6a3e74-1jammy python3-cephfs=19.0.0-2369-g1a6a3e74-1jammy python3-rbd=19.0.0-2369-g1a6a3e74-1jammy libcephfs2=19.0.0-2369-g1a6a3e74-1jammy libcephfs-dev=19.0.0-2369-g1a6a3e74-1jammy librados2=19.0.0-2369-g1a6a3e74-1jammy librbd1=19.0.0-2369-g1a6a3e74-1jammy rbd-fuse=19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:07:18.234 INFO:teuthology.orchestra.run.smithi120.stdout:Reading package lists... 2024-04-18T23:07:18.294 INFO:teuthology.orchestra.run.smithi142.stdout:Reading package lists... 2024-04-18T23:07:18.327 INFO:teuthology.orchestra.run.smithi073.stdout:Building dependency tree... 2024-04-18T23:07:18.328 INFO:teuthology.orchestra.run.smithi073.stdout:Reading state information... 2024-04-18T23:07:18.403 INFO:teuthology.orchestra.run.smithi136.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T23:07:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T23:07:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T23:07:18.403 INFO:teuthology.orchestra.run.smithi136.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T23:07:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T23:07:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T23:07:18.405 INFO:teuthology.orchestra.run.smithi136.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T23:07:18.405 INFO:teuthology.orchestra.run.smithi136.stdout:The following additional packages will be installed: 2024-04-18T23:07:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-18T23:07:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-18T23:07:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-18T23:07:18.405 INFO:teuthology.orchestra.run.smithi136.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-18T23:07:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-18T23:07:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-18T23:07:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-18T23:07:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-18T23:07:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-18T23:07:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T23:07:18.407 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: xmlstarlet zip 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout:Suggested packages: 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-18T23:07:18.408 INFO:teuthology.orchestra.run.smithi136.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-18T23:07:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-18T23:07:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-18T23:07:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-18T23:07:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-18T23:07:18.409 INFO:teuthology.orchestra.run.smithi136.stdout:Recommended packages: 2024-04-18T23:07:18.409 INFO:teuthology.orchestra.run.smithi136.stdout: btrfs-tools 2024-04-18T23:07:18.433 INFO:teuthology.orchestra.run.smithi120.stdout:Building dependency tree... 2024-04-18T23:07:18.434 INFO:teuthology.orchestra.run.smithi120.stdout:Reading state information... 2024-04-18T23:07:18.489 INFO:teuthology.orchestra.run.smithi142.stdout:Building dependency tree... 2024-04-18T23:07:18.490 INFO:teuthology.orchestra.run.smithi142.stdout:Reading state information... 2024-04-18T23:07:18.507 INFO:teuthology.orchestra.run.smithi136.stdout:The following NEW packages will be installed: 2024-04-18T23:07:18.507 INFO:teuthology.orchestra.run.smithi136.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-18T23:07:18.507 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-18T23:07:18.507 INFO:teuthology.orchestra.run.smithi136.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-18T23:07:18.512 INFO:teuthology.orchestra.run.smithi073.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T23:07:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T23:07:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T23:07:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T23:07:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T23:07:18.512 INFO:teuthology.orchestra.run.smithi073.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout:The following additional packages will be installed: 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-18T23:07:18.513 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-18T23:07:18.514 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet zip 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout:Suggested packages: 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout:Recommended packages: 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi073.stdout: btrfs-tools 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi136.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-18T23:07:18.515 INFO:teuthology.orchestra.run.smithi136.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T23:07:18.516 INFO:teuthology.orchestra.run.smithi136.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T23:07:18.517 INFO:teuthology.orchestra.run.smithi136.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-18T23:07:18.517 INFO:teuthology.orchestra.run.smithi136.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-18T23:07:18.517 INFO:teuthology.orchestra.run.smithi136.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-18T23:07:18.517 INFO:teuthology.orchestra.run.smithi136.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-18T23:07:18.517 INFO:teuthology.orchestra.run.smithi136.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-18T23:07:18.517 INFO:teuthology.orchestra.run.smithi136.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-18T23:07:18.517 INFO:teuthology.orchestra.run.smithi136.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-18T23:07:18.517 INFO:teuthology.orchestra.run.smithi136.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-18T23:07:18.577 INFO:teuthology.orchestra.run.smithi136.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-18T23:07:18.577 INFO:teuthology.orchestra.run.smithi136.stdout:Need to get 179 MB of archives. 2024-04-18T23:07:18.578 INFO:teuthology.orchestra.run.smithi136.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-18T23:07:18.578 INFO:teuthology.orchestra.run.smithi136.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-18T23:07:18.609 INFO:teuthology.orchestra.run.smithi073.stdout:The following NEW packages will be installed: 2024-04-18T23:07:18.609 INFO:teuthology.orchestra.run.smithi073.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-18T23:07:18.609 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-18T23:07:18.609 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-18T23:07:18.610 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-18T23:07:18.610 INFO:teuthology.orchestra.run.smithi073.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-18T23:07:18.610 INFO:teuthology.orchestra.run.smithi073.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-18T23:07:18.611 INFO:teuthology.orchestra.run.smithi073.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-18T23:07:18.611 INFO:teuthology.orchestra.run.smithi073.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-18T23:07:18.611 INFO:teuthology.orchestra.run.smithi073.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-18T23:07:18.611 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-18T23:07:18.611 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-18T23:07:18.611 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T23:07:18.611 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-18T23:07:18.612 INFO:teuthology.orchestra.run.smithi073.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-18T23:07:18.629 INFO:teuthology.orchestra.run.smithi120.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T23:07:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T23:07:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T23:07:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T23:07:18.629 INFO:teuthology.orchestra.run.smithi120.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T23:07:18.630 INFO:teuthology.orchestra.run.smithi120.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T23:07:18.630 INFO:teuthology.orchestra.run.smithi120.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T23:07:18.631 INFO:teuthology.orchestra.run.smithi120.stdout:The following additional packages will be installed: 2024-04-18T23:07:18.631 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T23:07:18.632 INFO:teuthology.orchestra.run.smithi120.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: xmlstarlet zip 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout:Suggested packages: 2024-04-18T23:07:18.633 INFO:teuthology.orchestra.run.smithi120.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-18T23:07:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-18T23:07:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-18T23:07:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-18T23:07:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-18T23:07:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-18T23:07:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-18T23:07:18.634 INFO:teuthology.orchestra.run.smithi120.stdout:Recommended packages: 2024-04-18T23:07:18.634 INFO:teuthology.orchestra.run.smithi120.stdout: btrfs-tools 2024-04-18T23:07:18.651 INFO:teuthology.orchestra.run.smithi136.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,547 kB] 2024-04-18T23:07:18.678 INFO:teuthology.orchestra.run.smithi073.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-18T23:07:18.678 INFO:teuthology.orchestra.run.smithi073.stdout:Need to get 179 MB of archives. 2024-04-18T23:07:18.678 INFO:teuthology.orchestra.run.smithi073.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-18T23:07:18.678 INFO:teuthology.orchestra.run.smithi073.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-18T23:07:18.683 INFO:teuthology.orchestra.run.smithi142.stdout:The following packages were automatically installed and are no longer required: 2024-04-18T23:07:18.683 INFO:teuthology.orchestra.run.smithi142.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-04-18T23:07:18.683 INFO:teuthology.orchestra.run.smithi142.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-04-18T23:07:18.683 INFO:teuthology.orchestra.run.smithi142.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-04-18T23:07:18.683 INFO:teuthology.orchestra.run.smithi142.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-04-18T23:07:18.684 INFO:teuthology.orchestra.run.smithi142.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-04-18T23:07:18.684 INFO:teuthology.orchestra.run.smithi142.stdout:Use 'sudo apt autoremove' to remove them. 2024-04-18T23:07:18.684 INFO:teuthology.orchestra.run.smithi136.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout:The following additional packages will be installed: 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T23:07:18.686 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-04-18T23:07:18.687 INFO:teuthology.orchestra.run.smithi142.stdout: xmlstarlet zip 2024-04-18T23:07:18.688 INFO:teuthology.orchestra.run.smithi136.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-18T23:07:18.688 INFO:teuthology.orchestra.run.smithi142.stdout:Suggested packages: 2024-04-18T23:07:18.688 INFO:teuthology.orchestra.run.smithi142.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-04-18T23:07:18.688 INFO:teuthology.orchestra.run.smithi142.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-04-18T23:07:18.688 INFO:teuthology.orchestra.run.smithi142.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-04-18T23:07:18.688 INFO:teuthology.orchestra.run.smithi142.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-04-18T23:07:18.688 INFO:teuthology.orchestra.run.smithi142.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-04-18T23:07:18.688 INFO:teuthology.orchestra.run.smithi142.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-04-18T23:07:18.689 INFO:teuthology.orchestra.run.smithi142.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-04-18T23:07:18.689 INFO:teuthology.orchestra.run.smithi142.stdout:Recommended packages: 2024-04-18T23:07:18.689 INFO:teuthology.orchestra.run.smithi142.stdout: btrfs-tools 2024-04-18T23:07:18.708 INFO:teuthology.orchestra.run.smithi136.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-18T23:07:18.734 INFO:teuthology.orchestra.run.smithi120.stdout:The following NEW packages will be installed: 2024-04-18T23:07:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-18T23:07:18.734 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-18T23:07:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-18T23:07:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-18T23:07:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-18T23:07:18.735 INFO:teuthology.orchestra.run.smithi120.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-18T23:07:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-18T23:07:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-18T23:07:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-18T23:07:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-18T23:07:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-18T23:07:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T23:07:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T23:07:18.736 INFO:teuthology.orchestra.run.smithi120.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-18T23:07:18.737 INFO:teuthology.orchestra.run.smithi120.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-18T23:07:18.752 INFO:teuthology.orchestra.run.smithi073.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,547 kB] 2024-04-18T23:07:18.777 INFO:teuthology.orchestra.run.smithi136.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-18T23:07:18.783 INFO:teuthology.orchestra.run.smithi136.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-18T23:07:18.788 INFO:teuthology.orchestra.run.smithi142.stdout:The following NEW packages will be installed: 2024-04-18T23:07:18.788 INFO:teuthology.orchestra.run.smithi142.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-04-18T23:07:18.788 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-04-18T23:07:18.788 INFO:teuthology.orchestra.run.smithi142.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-04-18T23:07:18.788 INFO:teuthology.orchestra.run.smithi142.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-04-18T23:07:18.788 INFO:teuthology.orchestra.run.smithi142.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-04-18T23:07:18.788 INFO:teuthology.orchestra.run.smithi142.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-04-18T23:07:18.789 INFO:teuthology.orchestra.run.smithi073.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-04-18T23:07:18.790 INFO:teuthology.orchestra.run.smithi142.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-04-18T23:07:18.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-04-18T23:07:18.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-04-18T23:07:18.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-04-18T23:07:18.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-04-18T23:07:18.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-04-18T23:07:18.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-04-18T23:07:18.791 INFO:teuthology.orchestra.run.smithi142.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-04-18T23:07:18.791 INFO:teuthology.orchestra.run.smithi142.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-04-18T23:07:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-18T23:07:18.795 INFO:teuthology.orchestra.run.smithi136.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-18T23:07:18.799 INFO:teuthology.orchestra.run.smithi120.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-18T23:07:18.799 INFO:teuthology.orchestra.run.smithi120.stdout:Need to get 179 MB of archives. 2024-04-18T23:07:18.799 INFO:teuthology.orchestra.run.smithi120.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-18T23:07:18.799 INFO:teuthology.orchestra.run.smithi120.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-18T23:07:18.800 INFO:teuthology.orchestra.run.smithi136.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-18T23:07:18.800 INFO:teuthology.orchestra.run.smithi136.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-18T23:07:18.800 INFO:teuthology.orchestra.run.smithi136.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-18T23:07:18.808 INFO:teuthology.orchestra.run.smithi136.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-18T23:07:18.810 INFO:teuthology.orchestra.run.smithi136.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-18T23:07:18.812 INFO:teuthology.orchestra.run.smithi073.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-18T23:07:18.813 INFO:teuthology.orchestra.run.smithi136.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-18T23:07:18.814 INFO:teuthology.orchestra.run.smithi136.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-18T23:07:18.825 INFO:teuthology.orchestra.run.smithi136.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-18T23:07:18.827 INFO:teuthology.orchestra.run.smithi136.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-18T23:07:18.829 INFO:teuthology.orchestra.run.smithi136.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-18T23:07:18.831 INFO:teuthology.orchestra.run.smithi136.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-18T23:07:18.846 INFO:teuthology.orchestra.run.smithi136.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-18T23:07:18.847 INFO:teuthology.orchestra.run.smithi136.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-18T23:07:18.848 INFO:teuthology.orchestra.run.smithi136.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-18T23:07:18.848 INFO:teuthology.orchestra.run.smithi136.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-18T23:07:18.848 INFO:teuthology.orchestra.run.smithi136.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-18T23:07:18.849 INFO:teuthology.orchestra.run.smithi136.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-18T23:07:18.854 INFO:teuthology.orchestra.run.smithi142.stdout:0 upgraded, 118 newly installed, 0 to remove and 302 not upgraded. 2024-04-18T23:07:18.854 INFO:teuthology.orchestra.run.smithi142.stdout:Need to get 179 MB of archives. 2024-04-18T23:07:18.854 INFO:teuthology.orchestra.run.smithi142.stdout:After this operation, 821 MB of additional disk space will be used. 2024-04-18T23:07:18.854 INFO:teuthology.orchestra.run.smithi142.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-04-18T23:07:18.868 INFO:teuthology.orchestra.run.smithi136.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-18T23:07:18.868 INFO:teuthology.orchestra.run.smithi136.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-18T23:07:18.868 INFO:teuthology.orchestra.run.smithi136.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-18T23:07:18.871 INFO:teuthology.orchestra.run.smithi136.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-18T23:07:18.885 INFO:teuthology.orchestra.run.smithi073.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-18T23:07:18.889 INFO:teuthology.orchestra.run.smithi136.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-18T23:07:18.890 INFO:teuthology.orchestra.run.smithi136.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-18T23:07:18.890 INFO:teuthology.orchestra.run.smithi073.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-18T23:07:18.890 INFO:teuthology.orchestra.run.smithi136.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-18T23:07:18.891 INFO:teuthology.orchestra.run.smithi136.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-18T23:07:18.892 INFO:teuthology.orchestra.run.smithi136.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-18T23:07:18.892 INFO:teuthology.orchestra.run.smithi136.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-18T23:07:18.902 INFO:teuthology.orchestra.run.smithi120.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-18T23:07:18.904 INFO:teuthology.orchestra.run.smithi073.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-18T23:07:18.906 INFO:teuthology.orchestra.run.smithi120.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-18T23:07:18.908 INFO:teuthology.orchestra.run.smithi073.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-18T23:07:18.908 INFO:teuthology.orchestra.run.smithi073.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-18T23:07:18.909 INFO:teuthology.orchestra.run.smithi073.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-18T23:07:18.912 INFO:teuthology.orchestra.run.smithi136.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-18T23:07:18.912 INFO:teuthology.orchestra.run.smithi136.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-18T23:07:18.913 INFO:teuthology.orchestra.run.smithi136.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-18T23:07:18.917 INFO:teuthology.orchestra.run.smithi073.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-18T23:07:18.918 INFO:teuthology.orchestra.run.smithi136.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-18T23:07:18.919 INFO:teuthology.orchestra.run.smithi073.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-18T23:07:18.921 INFO:teuthology.orchestra.run.smithi073.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-18T23:07:18.922 INFO:teuthology.orchestra.run.smithi073.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-18T23:07:18.925 INFO:teuthology.orchestra.run.smithi120.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-18T23:07:18.932 INFO:teuthology.orchestra.run.smithi136.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-18T23:07:18.933 INFO:teuthology.orchestra.run.smithi136.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-18T23:07:18.933 INFO:teuthology.orchestra.run.smithi136.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-18T23:07:18.933 INFO:teuthology.orchestra.run.smithi136.stdout:Get:43 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,168 kB] 2024-04-18T23:07:18.934 INFO:teuthology.orchestra.run.smithi136.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-18T23:07:18.936 INFO:teuthology.orchestra.run.smithi136.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-18T23:07:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-18T23:07:18.937 INFO:teuthology.orchestra.run.smithi136.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-18T23:07:18.938 INFO:teuthology.orchestra.run.smithi073.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-18T23:07:18.941 INFO:teuthology.orchestra.run.smithi073.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-18T23:07:18.943 INFO:teuthology.orchestra.run.smithi073.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-18T23:07:18.943 INFO:teuthology.orchestra.run.smithi073.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-18T23:07:18.953 INFO:teuthology.orchestra.run.smithi136.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-18T23:07:18.955 INFO:teuthology.orchestra.run.smithi136.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-18T23:07:18.957 INFO:teuthology.orchestra.run.smithi073.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-18T23:07:18.959 INFO:teuthology.orchestra.run.smithi073.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-18T23:07:18.959 INFO:teuthology.orchestra.run.smithi073.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-18T23:07:18.959 INFO:teuthology.orchestra.run.smithi136.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-18T23:07:18.959 INFO:teuthology.orchestra.run.smithi073.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-18T23:07:18.960 INFO:teuthology.orchestra.run.smithi073.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-18T23:07:18.960 INFO:teuthology.orchestra.run.smithi136.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-18T23:07:18.963 INFO:teuthology.orchestra.run.smithi142.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-04-18T23:07:18.966 INFO:teuthology.orchestra.run.smithi142.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-04-18T23:07:18.974 INFO:teuthology.orchestra.run.smithi136.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-18T23:07:18.979 INFO:teuthology.orchestra.run.smithi136.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-18T23:07:18.987 INFO:teuthology.orchestra.run.smithi073.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-18T23:07:18.987 INFO:teuthology.orchestra.run.smithi073.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-18T23:07:18.988 INFO:teuthology.orchestra.run.smithi073.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-18T23:07:18.990 INFO:teuthology.orchestra.run.smithi120.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,547 kB] 2024-04-18T23:07:18.992 INFO:teuthology.orchestra.run.smithi073.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-18T23:07:18.993 INFO:teuthology.orchestra.run.smithi073.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-18T23:07:19.009 INFO:teuthology.orchestra.run.smithi142.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-04-18T23:07:19.016 INFO:teuthology.orchestra.run.smithi120.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-18T23:07:19.021 INFO:teuthology.orchestra.run.smithi120.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-18T23:07:19.030 INFO:teuthology.orchestra.run.smithi073.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-18T23:07:19.031 INFO:teuthology.orchestra.run.smithi073.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-18T23:07:19.033 INFO:teuthology.orchestra.run.smithi073.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-18T23:07:19.033 INFO:teuthology.orchestra.run.smithi073.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-18T23:07:19.034 INFO:teuthology.orchestra.run.smithi073.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-18T23:07:19.037 INFO:teuthology.orchestra.run.smithi120.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-18T23:07:19.042 INFO:teuthology.orchestra.run.smithi120.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-18T23:07:19.042 INFO:teuthology.orchestra.run.smithi120.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-18T23:07:19.043 INFO:teuthology.orchestra.run.smithi120.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-18T23:07:19.053 INFO:teuthology.orchestra.run.smithi120.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-18T23:07:19.056 INFO:teuthology.orchestra.run.smithi136.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-18T23:07:19.056 INFO:teuthology.orchestra.run.smithi120.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-18T23:07:19.057 INFO:teuthology.orchestra.run.smithi136.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-18T23:07:19.058 INFO:teuthology.orchestra.run.smithi136.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-18T23:07:19.059 INFO:teuthology.orchestra.run.smithi120.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-18T23:07:19.059 INFO:teuthology.orchestra.run.smithi136.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-18T23:07:19.059 INFO:teuthology.orchestra.run.smithi073.stdout:Get:36 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,168 kB] 2024-04-18T23:07:19.060 INFO:teuthology.orchestra.run.smithi136.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-18T23:07:19.061 INFO:teuthology.orchestra.run.smithi120.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-18T23:07:19.069 INFO:teuthology.orchestra.run.smithi073.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-18T23:07:19.071 INFO:teuthology.orchestra.run.smithi073.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-18T23:07:19.072 INFO:teuthology.orchestra.run.smithi073.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-18T23:07:19.083 INFO:teuthology.orchestra.run.smithi073.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-18T23:07:19.083 INFO:teuthology.orchestra.run.smithi073.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-18T23:07:19.085 INFO:teuthology.orchestra.run.smithi136.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-18T23:07:19.088 INFO:teuthology.orchestra.run.smithi136.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-18T23:07:19.088 INFO:teuthology.orchestra.run.smithi136.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-18T23:07:19.089 INFO:teuthology.orchestra.run.smithi142.stdout:Get:5 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,547 kB] 2024-04-18T23:07:19.097 INFO:teuthology.orchestra.run.smithi120.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-18T23:07:19.100 INFO:teuthology.orchestra.run.smithi120.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-18T23:07:19.101 INFO:teuthology.orchestra.run.smithi136.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2369-g1a6a3e74-1jammy [711 kB] 2024-04-18T23:07:19.103 INFO:teuthology.orchestra.run.smithi120.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-18T23:07:19.106 INFO:teuthology.orchestra.run.smithi120.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-18T23:07:19.106 INFO:teuthology.orchestra.run.smithi120.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-18T23:07:19.107 INFO:teuthology.orchestra.run.smithi120.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-18T23:07:19.109 INFO:teuthology.orchestra.run.smithi120.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-18T23:07:19.109 INFO:teuthology.orchestra.run.smithi120.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-18T23:07:19.109 INFO:teuthology.orchestra.run.smithi120.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-18T23:07:19.111 INFO:teuthology.orchestra.run.smithi073.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-18T23:07:19.112 INFO:teuthology.orchestra.run.smithi073.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-18T23:07:19.115 INFO:teuthology.orchestra.run.smithi073.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-18T23:07:19.116 INFO:teuthology.orchestra.run.smithi136.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-18T23:07:19.116 INFO:teuthology.orchestra.run.smithi136.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-18T23:07:19.119 INFO:teuthology.orchestra.run.smithi073.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-18T23:07:19.121 INFO:teuthology.orchestra.run.smithi073.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-18T23:07:19.123 INFO:teuthology.orchestra.run.smithi136.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-18T23:07:19.124 INFO:teuthology.orchestra.run.smithi136.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-18T23:07:19.124 INFO:teuthology.orchestra.run.smithi136.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-18T23:07:19.125 INFO:teuthology.orchestra.run.smithi136.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-18T23:07:19.126 INFO:teuthology.orchestra.run.smithi136.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-18T23:07:19.126 INFO:teuthology.orchestra.run.smithi136.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-18T23:07:19.129 INFO:teuthology.orchestra.run.smithi120.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-18T23:07:19.131 INFO:teuthology.orchestra.run.smithi136.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-18T23:07:19.131 INFO:teuthology.orchestra.run.smithi120.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-18T23:07:19.132 INFO:teuthology.orchestra.run.smithi120.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-18T23:07:19.132 INFO:teuthology.orchestra.run.smithi120.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-18T23:07:19.134 INFO:teuthology.orchestra.run.smithi073.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-18T23:07:19.136 INFO:teuthology.orchestra.run.smithi073.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-18T23:07:19.136 INFO:teuthology.orchestra.run.smithi120.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-18T23:07:19.137 INFO:teuthology.orchestra.run.smithi142.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-04-18T23:07:19.138 INFO:teuthology.orchestra.run.smithi120.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-18T23:07:19.138 INFO:teuthology.orchestra.run.smithi120.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-18T23:07:19.138 INFO:teuthology.orchestra.run.smithi120.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-18T23:07:19.139 INFO:teuthology.orchestra.run.smithi120.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-18T23:07:19.141 INFO:teuthology.orchestra.run.smithi073.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-18T23:07:19.142 INFO:teuthology.orchestra.run.smithi073.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-18T23:07:19.143 INFO:teuthology.orchestra.run.smithi073.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-18T23:07:19.144 INFO:teuthology.orchestra.run.smithi142.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-04-18T23:07:19.145 INFO:teuthology.orchestra.run.smithi136.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-18T23:07:19.147 INFO:teuthology.orchestra.run.smithi136.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-18T23:07:19.148 INFO:teuthology.orchestra.run.smithi136.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-18T23:07:19.149 INFO:teuthology.orchestra.run.smithi136.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2369-g1a6a3e74-1jammy [356 kB] 2024-04-18T23:07:19.152 INFO:teuthology.orchestra.run.smithi120.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-18T23:07:19.156 INFO:teuthology.orchestra.run.smithi073.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-18T23:07:19.159 INFO:teuthology.orchestra.run.smithi142.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-04-18T23:07:19.164 INFO:teuthology.orchestra.run.smithi142.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-04-18T23:07:19.165 INFO:teuthology.orchestra.run.smithi142.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-04-18T23:07:19.166 INFO:teuthology.orchestra.run.smithi142.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-04-18T23:07:19.173 INFO:teuthology.orchestra.run.smithi120.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-18T23:07:19.176 INFO:teuthology.orchestra.run.smithi120.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-18T23:07:19.178 INFO:teuthology.orchestra.run.smithi120.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-18T23:07:19.178 INFO:teuthology.orchestra.run.smithi120.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-18T23:07:19.186 INFO:teuthology.orchestra.run.smithi136.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2369-g1a6a3e74-1jammy [32.8 kB] 2024-04-18T23:07:19.187 INFO:teuthology.orchestra.run.smithi136.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2369-g1a6a3e74-1jammy [180 kB] 2024-04-18T23:07:19.187 INFO:teuthology.orchestra.run.smithi142.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-04-18T23:07:19.192 INFO:teuthology.orchestra.run.smithi120.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-18T23:07:19.194 INFO:teuthology.orchestra.run.smithi136.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2369-g1a6a3e74-1jammy [65.7 kB] 2024-04-18T23:07:19.194 INFO:teuthology.orchestra.run.smithi136.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-18T23:07:19.194 INFO:teuthology.orchestra.run.smithi120.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-18T23:07:19.194 INFO:teuthology.orchestra.run.smithi120.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-18T23:07:19.195 INFO:teuthology.orchestra.run.smithi136.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2369-g1a6a3e74-1jammy [330 kB] 2024-04-18T23:07:19.195 INFO:teuthology.orchestra.run.smithi120.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-18T23:07:19.197 INFO:teuthology.orchestra.run.smithi142.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-04-18T23:07:19.198 INFO:teuthology.orchestra.run.smithi136.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-18T23:07:19.199 INFO:teuthology.orchestra.run.smithi136.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-18T23:07:19.200 INFO:teuthology.orchestra.run.smithi136.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-18T23:07:19.203 INFO:teuthology.orchestra.run.smithi142.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-04-18T23:07:19.206 INFO:teuthology.orchestra.run.smithi136.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-18T23:07:19.207 INFO:teuthology.orchestra.run.smithi136.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-18T23:07:19.207 INFO:teuthology.orchestra.run.smithi142.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-04-18T23:07:19.208 INFO:teuthology.orchestra.run.smithi120.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-18T23:07:19.211 INFO:teuthology.orchestra.run.smithi136.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-18T23:07:19.229 INFO:teuthology.orchestra.run.smithi136.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-18T23:07:19.231 INFO:teuthology.orchestra.run.smithi136.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-18T23:07:19.231 INFO:teuthology.orchestra.run.smithi136.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-18T23:07:19.232 INFO:teuthology.orchestra.run.smithi142.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-04-18T23:07:19.232 INFO:teuthology.orchestra.run.smithi136.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2369-g1a6a3e74-1jammy [6,775 kB] 2024-04-18T23:07:19.233 INFO:teuthology.orchestra.run.smithi120.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-18T23:07:19.235 INFO:teuthology.orchestra.run.smithi142.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-04-18T23:07:19.235 INFO:teuthology.orchestra.run.smithi120.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-18T23:07:19.235 INFO:teuthology.orchestra.run.smithi073.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-18T23:07:19.236 INFO:teuthology.orchestra.run.smithi073.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-18T23:07:19.237 INFO:teuthology.orchestra.run.smithi120.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-18T23:07:19.238 INFO:teuthology.orchestra.run.smithi142.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-04-18T23:07:19.239 INFO:teuthology.orchestra.run.smithi120.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-18T23:07:19.240 INFO:teuthology.orchestra.run.smithi142.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-04-18T23:07:19.241 INFO:teuthology.orchestra.run.smithi142.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-04-18T23:07:19.242 INFO:teuthology.orchestra.run.smithi142.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-04-18T23:07:19.244 INFO:teuthology.orchestra.run.smithi120.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-18T23:07:19.246 INFO:teuthology.orchestra.run.smithi120.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-18T23:07:19.247 INFO:teuthology.orchestra.run.smithi120.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-18T23:07:19.248 INFO:teuthology.orchestra.run.smithi073.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-18T23:07:19.248 INFO:teuthology.orchestra.run.smithi073.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-18T23:07:19.249 INFO:teuthology.orchestra.run.smithi073.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-18T23:07:19.252 INFO:teuthology.orchestra.run.smithi120.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-18T23:07:19.278 INFO:teuthology.orchestra.run.smithi073.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2369-g1a6a3e74-1jammy [711 kB] 2024-04-18T23:07:19.282 INFO:teuthology.orchestra.run.smithi073.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-18T23:07:19.285 INFO:teuthology.orchestra.run.smithi073.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-18T23:07:19.285 INFO:teuthology.orchestra.run.smithi073.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-18T23:07:19.285 INFO:teuthology.orchestra.run.smithi136.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-18T23:07:19.286 INFO:teuthology.orchestra.run.smithi136.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-18T23:07:19.289 INFO:teuthology.orchestra.run.smithi142.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-04-18T23:07:19.289 INFO:teuthology.orchestra.run.smithi142.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-04-18T23:07:19.290 INFO:teuthology.orchestra.run.smithi142.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-04-18T23:07:19.290 INFO:teuthology.orchestra.run.smithi142.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-04-18T23:07:19.298 INFO:teuthology.orchestra.run.smithi136.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-18T23:07:19.299 INFO:teuthology.orchestra.run.smithi136.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-18T23:07:19.299 INFO:teuthology.orchestra.run.smithi136.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-18T23:07:19.303 INFO:teuthology.orchestra.run.smithi136.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-18T23:07:19.304 INFO:teuthology.orchestra.run.smithi136.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-18T23:07:19.332 INFO:teuthology.orchestra.run.smithi073.stdout:Get:62 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2369-g1a6a3e74-1jammy [356 kB] 2024-04-18T23:07:19.338 INFO:teuthology.orchestra.run.smithi142.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-04-18T23:07:19.339 INFO:teuthology.orchestra.run.smithi142.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-04-18T23:07:19.340 INFO:teuthology.orchestra.run.smithi142.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-04-18T23:07:19.345 INFO:teuthology.orchestra.run.smithi142.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-04-18T23:07:19.347 INFO:teuthology.orchestra.run.smithi142.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-04-18T23:07:19.348 INFO:teuthology.orchestra.run.smithi142.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-04-18T23:07:19.371 INFO:teuthology.orchestra.run.smithi142.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-04-18T23:07:19.372 INFO:teuthology.orchestra.run.smithi073.stdout:Get:63 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2369-g1a6a3e74-1jammy [32.8 kB] 2024-04-18T23:07:19.373 INFO:teuthology.orchestra.run.smithi142.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-04-18T23:07:19.373 INFO:teuthology.orchestra.run.smithi073.stdout:Get:64 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2369-g1a6a3e74-1jammy [180 kB] 2024-04-18T23:07:19.374 INFO:teuthology.orchestra.run.smithi142.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-04-18T23:07:19.375 INFO:teuthology.orchestra.run.smithi142.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-04-18T23:07:19.378 INFO:teuthology.orchestra.run.smithi073.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-18T23:07:19.379 INFO:teuthology.orchestra.run.smithi073.stdout:Get:66 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2369-g1a6a3e74-1jammy [65.7 kB] 2024-04-18T23:07:19.379 INFO:teuthology.orchestra.run.smithi073.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-18T23:07:19.380 INFO:teuthology.orchestra.run.smithi073.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-18T23:07:19.380 INFO:teuthology.orchestra.run.smithi073.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-18T23:07:19.381 INFO:teuthology.orchestra.run.smithi073.stdout:Get:70 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2369-g1a6a3e74-1jammy [330 kB] 2024-04-18T23:07:19.381 INFO:teuthology.orchestra.run.smithi073.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-18T23:07:19.382 INFO:teuthology.orchestra.run.smithi073.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-18T23:07:19.382 INFO:teuthology.orchestra.run.smithi073.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-18T23:07:19.383 INFO:teuthology.orchestra.run.smithi073.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-18T23:07:19.388 INFO:teuthology.orchestra.run.smithi073.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-18T23:07:19.391 INFO:teuthology.orchestra.run.smithi142.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-04-18T23:07:19.393 INFO:teuthology.orchestra.run.smithi142.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-04-18T23:07:19.394 INFO:teuthology.orchestra.run.smithi142.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-04-18T23:07:19.397 INFO:teuthology.orchestra.run.smithi073.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2369-g1a6a3e74-1jammy [6,775 kB] 2024-04-18T23:07:19.404 INFO:teuthology.orchestra.run.smithi073.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-18T23:07:19.404 INFO:teuthology.orchestra.run.smithi120.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-18T23:07:19.405 INFO:teuthology.orchestra.run.smithi120.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-18T23:07:19.405 INFO:teuthology.orchestra.run.smithi142.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-04-18T23:07:19.406 INFO:teuthology.orchestra.run.smithi142.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-04-18T23:07:19.407 INFO:teuthology.orchestra.run.smithi120.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-18T23:07:19.408 INFO:teuthology.orchestra.run.smithi142.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-04-18T23:07:19.408 INFO:teuthology.orchestra.run.smithi120.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-18T23:07:19.410 INFO:teuthology.orchestra.run.smithi120.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-18T23:07:19.411 INFO:teuthology.orchestra.run.smithi142.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-04-18T23:07:19.413 INFO:teuthology.orchestra.run.smithi073.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-18T23:07:19.414 INFO:teuthology.orchestra.run.smithi073.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-18T23:07:19.414 INFO:teuthology.orchestra.run.smithi142.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-04-18T23:07:19.418 INFO:teuthology.orchestra.run.smithi142.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-04-18T23:07:19.420 INFO:teuthology.orchestra.run.smithi073.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-18T23:07:19.420 INFO:teuthology.orchestra.run.smithi142.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-04-18T23:07:19.424 INFO:teuthology.orchestra.run.smithi073.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-18T23:07:19.425 INFO:teuthology.orchestra.run.smithi073.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-18T23:07:19.426 INFO:teuthology.orchestra.run.smithi073.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-18T23:07:19.433 INFO:teuthology.orchestra.run.smithi073.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-18T23:07:19.434 INFO:teuthology.orchestra.run.smithi120.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-18T23:07:19.434 INFO:teuthology.orchestra.run.smithi073.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-18T23:07:19.436 INFO:teuthology.orchestra.run.smithi120.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-18T23:07:19.436 INFO:teuthology.orchestra.run.smithi120.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-18T23:07:19.439 INFO:teuthology.orchestra.run.smithi073.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-18T23:07:19.440 INFO:teuthology.orchestra.run.smithi142.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-04-18T23:07:19.442 INFO:teuthology.orchestra.run.smithi142.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-04-18T23:07:19.448 INFO:teuthology.orchestra.run.smithi142.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-04-18T23:07:19.449 INFO:teuthology.orchestra.run.smithi142.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-04-18T23:07:19.451 INFO:teuthology.orchestra.run.smithi142.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-04-18T23:07:19.453 INFO:teuthology.orchestra.run.smithi120.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-18T23:07:19.453 INFO:teuthology.orchestra.run.smithi120.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-18T23:07:19.458 INFO:teuthology.orchestra.run.smithi142.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-04-18T23:07:19.469 INFO:teuthology.orchestra.run.smithi073.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-18T23:07:19.470 INFO:teuthology.orchestra.run.smithi073.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-18T23:07:19.471 INFO:teuthology.orchestra.run.smithi073.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-18T23:07:19.473 INFO:teuthology.orchestra.run.smithi073.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-18T23:07:19.475 INFO:teuthology.orchestra.run.smithi073.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-18T23:07:19.479 INFO:teuthology.orchestra.run.smithi120.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-18T23:07:19.480 INFO:teuthology.orchestra.run.smithi120.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-18T23:07:19.480 INFO:teuthology.orchestra.run.smithi120.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-18T23:07:19.481 INFO:teuthology.orchestra.run.smithi120.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-18T23:07:19.481 INFO:teuthology.orchestra.run.smithi120.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-18T23:07:19.485 INFO:teuthology.orchestra.run.smithi073.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-18T23:07:19.485 INFO:teuthology.orchestra.run.smithi073.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-18T23:07:19.495 INFO:teuthology.orchestra.run.smithi073.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-18T23:07:19.499 INFO:teuthology.orchestra.run.smithi073.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-18T23:07:19.500 INFO:teuthology.orchestra.run.smithi073.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-18T23:07:19.519 INFO:teuthology.orchestra.run.smithi120.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-18T23:07:19.524 INFO:teuthology.orchestra.run.smithi120.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-18T23:07:19.526 INFO:teuthology.orchestra.run.smithi120.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-18T23:07:19.528 INFO:teuthology.orchestra.run.smithi120.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-18T23:07:19.529 INFO:teuthology.orchestra.run.smithi120.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-18T23:07:19.534 INFO:teuthology.orchestra.run.smithi142.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-04-18T23:07:19.536 INFO:teuthology.orchestra.run.smithi142.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-04-18T23:07:19.538 INFO:teuthology.orchestra.run.smithi142.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-04-18T23:07:19.538 INFO:teuthology.orchestra.run.smithi142.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-04-18T23:07:19.540 INFO:teuthology.orchestra.run.smithi142.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-04-18T23:07:19.553 INFO:teuthology.orchestra.run.smithi120.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-18T23:07:19.558 INFO:teuthology.orchestra.run.smithi120.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-18T23:07:19.560 INFO:teuthology.orchestra.run.smithi120.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-18T23:07:19.561 INFO:teuthology.orchestra.run.smithi120.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-18T23:07:19.566 INFO:teuthology.orchestra.run.smithi142.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-04-18T23:07:19.568 INFO:teuthology.orchestra.run.smithi142.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-04-18T23:07:19.569 INFO:teuthology.orchestra.run.smithi142.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-04-18T23:07:19.570 INFO:teuthology.orchestra.run.smithi120.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-18T23:07:19.581 INFO:teuthology.orchestra.run.smithi120.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-18T23:07:19.587 INFO:teuthology.orchestra.run.smithi142.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-04-18T23:07:19.587 INFO:teuthology.orchestra.run.smithi142.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-04-18T23:07:19.588 INFO:teuthology.orchestra.run.smithi120.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-18T23:07:19.591 INFO:teuthology.orchestra.run.smithi120.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-18T23:07:19.591 INFO:teuthology.orchestra.run.smithi142.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-04-18T23:07:19.592 INFO:teuthology.orchestra.run.smithi142.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-04-18T23:07:19.592 INFO:teuthology.orchestra.run.smithi142.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-04-18T23:07:19.592 INFO:teuthology.orchestra.run.smithi142.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-04-18T23:07:19.593 INFO:teuthology.orchestra.run.smithi120.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-18T23:07:19.593 INFO:teuthology.orchestra.run.smithi142.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-04-18T23:07:19.595 INFO:teuthology.orchestra.run.smithi120.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-18T23:07:19.601 INFO:teuthology.orchestra.run.smithi120.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-18T23:07:19.606 INFO:teuthology.orchestra.run.smithi120.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-18T23:07:19.611 INFO:teuthology.orchestra.run.smithi142.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-04-18T23:07:19.615 INFO:teuthology.orchestra.run.smithi142.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-04-18T23:07:19.617 INFO:teuthology.orchestra.run.smithi142.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-04-18T23:07:19.618 INFO:teuthology.orchestra.run.smithi120.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-18T23:07:19.618 INFO:teuthology.orchestra.run.smithi120.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-18T23:07:19.619 INFO:teuthology.orchestra.run.smithi142.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-04-18T23:07:19.619 INFO:teuthology.orchestra.run.smithi120.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-18T23:07:19.619 INFO:teuthology.orchestra.run.smithi142.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-04-18T23:07:19.623 INFO:teuthology.orchestra.run.smithi120.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-18T23:07:19.624 INFO:teuthology.orchestra.run.smithi120.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-18T23:07:19.639 INFO:teuthology.orchestra.run.smithi142.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-04-18T23:07:19.642 INFO:teuthology.orchestra.run.smithi142.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-04-18T23:07:19.643 INFO:teuthology.orchestra.run.smithi142.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-04-18T23:07:19.644 INFO:teuthology.orchestra.run.smithi142.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-04-18T23:07:19.649 INFO:teuthology.orchestra.run.smithi142.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-04-18T23:07:19.659 INFO:teuthology.orchestra.run.smithi142.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-04-18T23:07:19.663 INFO:teuthology.orchestra.run.smithi142.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-04-18T23:07:19.663 INFO:teuthology.orchestra.run.smithi142.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-04-18T23:07:19.664 INFO:teuthology.orchestra.run.smithi142.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-04-18T23:07:19.665 INFO:teuthology.orchestra.run.smithi142.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-04-18T23:07:19.679 INFO:teuthology.orchestra.run.smithi142.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-04-18T23:07:19.681 INFO:teuthology.orchestra.run.smithi142.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-04-18T23:07:19.688 INFO:teuthology.orchestra.run.smithi142.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-04-18T23:07:19.689 INFO:teuthology.orchestra.run.smithi142.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-04-18T23:07:19.689 INFO:teuthology.orchestra.run.smithi142.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-04-18T23:07:19.699 INFO:teuthology.orchestra.run.smithi142.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-04-18T23:07:19.700 INFO:teuthology.orchestra.run.smithi142.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-04-18T23:07:19.723 INFO:teuthology.orchestra.run.smithi136.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2369-g1a6a3e74-1jammy [112 kB] 2024-04-18T23:07:19.726 INFO:teuthology.orchestra.run.smithi136.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2369-g1a6a3e74-1jammy [466 kB] 2024-04-18T23:07:19.735 INFO:teuthology.orchestra.run.smithi136.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2369-g1a6a3e74-1jammy [25.6 MB] 2024-04-18T23:07:19.914 INFO:teuthology.orchestra.run.smithi073.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2369-g1a6a3e74-1jammy [112 kB] 2024-04-18T23:07:19.936 INFO:teuthology.orchestra.run.smithi073.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2369-g1a6a3e74-1jammy [466 kB] 2024-04-18T23:07:19.954 INFO:teuthology.orchestra.run.smithi073.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2369-g1a6a3e74-1jammy [25.6 MB] 2024-04-18T23:07:21.029 INFO:teuthology.orchestra.run.smithi120.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,168 kB] 2024-04-18T23:07:21.530 INFO:teuthology.orchestra.run.smithi142.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-2369-g1a6a3e74-1jammy [3,168 kB] 2024-04-18T23:07:21.708 INFO:teuthology.orchestra.run.smithi136.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2369-g1a6a3e74-1jammy [5,084 kB] 2024-04-18T23:07:21.873 INFO:teuthology.orchestra.run.smithi120.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2369-g1a6a3e74-1jammy [711 kB] 2024-04-18T23:07:22.003 INFO:teuthology.orchestra.run.smithi136.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2369-g1a6a3e74-1jammy [244 kB] 2024-04-18T23:07:22.014 INFO:teuthology.orchestra.run.smithi120.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2369-g1a6a3e74-1jammy [356 kB] 2024-04-18T23:07:22.031 INFO:teuthology.orchestra.run.smithi136.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [124 kB] 2024-04-18T23:07:22.036 INFO:teuthology.orchestra.run.smithi136.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2369-g1a6a3e74-1jammy [1,525 kB] 2024-04-18T23:07:22.091 INFO:teuthology.orchestra.run.smithi120.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2369-g1a6a3e74-1jammy [32.8 kB] 2024-04-18T23:07:22.099 INFO:teuthology.orchestra.run.smithi120.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2369-g1a6a3e74-1jammy [180 kB] 2024-04-18T23:07:22.106 INFO:teuthology.orchestra.run.smithi073.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2369-g1a6a3e74-1jammy [5,084 kB] 2024-04-18T23:07:22.139 INFO:teuthology.orchestra.run.smithi136.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2369-g1a6a3e74-1jammy [6,246 kB] 2024-04-18T23:07:22.149 INFO:teuthology.orchestra.run.smithi120.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2369-g1a6a3e74-1jammy [65.7 kB] 2024-04-18T23:07:22.154 INFO:teuthology.orchestra.run.smithi120.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2369-g1a6a3e74-1jammy [330 kB] 2024-04-18T23:07:22.242 INFO:teuthology.orchestra.run.smithi120.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2369-g1a6a3e74-1jammy [6,775 kB] 2024-04-18T23:07:22.457 INFO:teuthology.orchestra.run.smithi073.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2369-g1a6a3e74-1jammy [244 kB] 2024-04-18T23:07:22.499 INFO:teuthology.orchestra.run.smithi073.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [124 kB] 2024-04-18T23:07:22.503 INFO:teuthology.orchestra.run.smithi073.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2369-g1a6a3e74-1jammy [1,525 kB] 2024-04-18T23:07:22.629 INFO:teuthology.orchestra.run.smithi073.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2369-g1a6a3e74-1jammy [6,246 kB] 2024-04-18T23:07:22.654 INFO:teuthology.orchestra.run.smithi136.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2369-g1a6a3e74-1jammy [22.7 MB] 2024-04-18T23:07:23.085 INFO:teuthology.orchestra.run.smithi142.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-2369-g1a6a3e74-1jammy [711 kB] 2024-04-18T23:07:23.177 INFO:teuthology.orchestra.run.smithi073.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2369-g1a6a3e74-1jammy [22.7 MB] 2024-04-18T23:07:23.420 INFO:teuthology.orchestra.run.smithi142.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-2369-g1a6a3e74-1jammy [356 kB] 2024-04-18T23:07:23.584 INFO:teuthology.orchestra.run.smithi142.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-2369-g1a6a3e74-1jammy [32.8 kB] 2024-04-18T23:07:23.635 INFO:teuthology.orchestra.run.smithi142.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-2369-g1a6a3e74-1jammy [180 kB] 2024-04-18T23:07:23.729 INFO:teuthology.orchestra.run.smithi142.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-2369-g1a6a3e74-1jammy [65.7 kB] 2024-04-18T23:07:23.739 INFO:teuthology.orchestra.run.smithi142.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-2369-g1a6a3e74-1jammy [330 kB] 2024-04-18T23:07:23.804 INFO:teuthology.orchestra.run.smithi120.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2369-g1a6a3e74-1jammy [112 kB] 2024-04-18T23:07:23.846 INFO:teuthology.orchestra.run.smithi120.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2369-g1a6a3e74-1jammy [466 kB] 2024-04-18T23:07:23.901 INFO:teuthology.orchestra.run.smithi142.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-2369-g1a6a3e74-1jammy [6,775 kB] 2024-04-18T23:07:23.940 INFO:teuthology.orchestra.run.smithi120.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2369-g1a6a3e74-1jammy [25.6 MB] 2024-04-18T23:07:24.564 INFO:teuthology.orchestra.run.smithi136.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [14.0 kB] 2024-04-18T23:07:24.565 INFO:teuthology.orchestra.run.smithi136.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [869 kB] 2024-04-18T23:07:24.582 INFO:teuthology.orchestra.run.smithi136.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2369-g1a6a3e74-1jammy [2,435 kB] 2024-04-18T23:07:24.748 INFO:teuthology.orchestra.run.smithi136.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2369-g1a6a3e74-1jammy [791 kB] 2024-04-18T23:07:24.810 INFO:teuthology.orchestra.run.smithi136.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2369-g1a6a3e74-1jammy [147 kB] 2024-04-18T23:07:24.834 INFO:teuthology.orchestra.run.smithi136.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2369-g1a6a3e74-1jammy [2,295 kB] 2024-04-18T23:07:25.010 INFO:teuthology.orchestra.run.smithi136.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2369-g1a6a3e74-1jammy [8,625 kB] 2024-04-18T23:07:25.238 INFO:teuthology.orchestra.run.smithi073.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [14.0 kB] 2024-04-18T23:07:25.241 INFO:teuthology.orchestra.run.smithi073.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [869 kB] 2024-04-18T23:07:25.258 INFO:teuthology.orchestra.run.smithi073.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2369-g1a6a3e74-1jammy [2,435 kB] 2024-04-18T23:07:25.441 INFO:teuthology.orchestra.run.smithi073.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2369-g1a6a3e74-1jammy [791 kB] 2024-04-18T23:07:25.507 INFO:teuthology.orchestra.run.smithi073.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2369-g1a6a3e74-1jammy [147 kB] 2024-04-18T23:07:25.514 INFO:teuthology.orchestra.run.smithi073.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2369-g1a6a3e74-1jammy [2,295 kB] 2024-04-18T23:07:25.692 INFO:teuthology.orchestra.run.smithi136.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2369-g1a6a3e74-1jammy [14.2 kB] 2024-04-18T23:07:25.692 INFO:teuthology.orchestra.run.smithi136.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2369-g1a6a3e74-1jammy [50.7 MB] 2024-04-18T23:07:25.719 INFO:teuthology.orchestra.run.smithi073.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2369-g1a6a3e74-1jammy [8,625 kB] 2024-04-18T23:07:26.478 INFO:teuthology.orchestra.run.smithi073.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2369-g1a6a3e74-1jammy [14.2 kB] 2024-04-18T23:07:26.479 INFO:teuthology.orchestra.run.smithi073.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2369-g1a6a3e74-1jammy [50.7 MB] 2024-04-18T23:07:27.374 INFO:teuthology.orchestra.run.smithi142.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-2369-g1a6a3e74-1jammy [112 kB] 2024-04-18T23:07:27.396 INFO:teuthology.orchestra.run.smithi142.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-2369-g1a6a3e74-1jammy [466 kB] 2024-04-18T23:07:27.636 INFO:teuthology.orchestra.run.smithi142.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-2369-g1a6a3e74-1jammy [25.6 MB] 2024-04-18T23:07:29.574 INFO:teuthology.orchestra.run.smithi120.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2369-g1a6a3e74-1jammy [5,084 kB] 2024-04-18T23:07:30.077 INFO:teuthology.orchestra.run.smithi136.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2369-g1a6a3e74-1jammy [121 kB] 2024-04-18T23:07:30.079 INFO:teuthology.orchestra.run.smithi136.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2369-g1a6a3e74-1jammy [38.9 kB] 2024-04-18T23:07:30.080 INFO:teuthology.orchestra.run.smithi136.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2369-g1a6a3e74-1jammy [13.4 MB] 2024-04-18T23:07:30.488 INFO:teuthology.orchestra.run.smithi120.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2369-g1a6a3e74-1jammy [244 kB] 2024-04-18T23:07:30.552 INFO:teuthology.orchestra.run.smithi120.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [124 kB] 2024-04-18T23:07:30.565 INFO:teuthology.orchestra.run.smithi120.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2369-g1a6a3e74-1jammy [1,525 kB] 2024-04-18T23:07:30.905 INFO:teuthology.orchestra.run.smithi120.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2369-g1a6a3e74-1jammy [6,246 kB] 2024-04-18T23:07:31.061 INFO:teuthology.orchestra.run.smithi136.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [91.4 kB] 2024-04-18T23:07:31.239 INFO:teuthology.orchestra.run.smithi073.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2369-g1a6a3e74-1jammy [121 kB] 2024-04-18T23:07:31.241 INFO:teuthology.orchestra.run.smithi073.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2369-g1a6a3e74-1jammy [38.9 kB] 2024-04-18T23:07:31.242 INFO:teuthology.orchestra.run.smithi073.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2369-g1a6a3e74-1jammy [13.4 MB] 2024-04-18T23:07:31.612 INFO:teuthology.orchestra.run.smithi136.stdout:Fetched 179 MB in 13s (14.3 MB/s) 2024-04-18T23:07:31.757 INFO:teuthology.orchestra.run.smithi120.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2369-g1a6a3e74-1jammy [22.7 MB] 2024-04-18T23:07:31.785 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-18T23:07:31.791 INFO:teuthology.orchestra.run.smithi073.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [91.4 kB] 2024-04-18T23:07:31.885 INFO:teuthology.orchestra.run.smithi136.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-18T23:07:31.891 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-18T23:07:31.975 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T23:07:32.270 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-18T23:07:32.296 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-18T23:07:32.336 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T23:07:32.350 INFO:teuthology.orchestra.run.smithi073.stdout:Fetched 179 MB in 13s (13.6 MB/s) 2024-04-18T23:07:32.509 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-18T23:07:32.609 INFO:teuthology.orchestra.run.smithi073.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-18T23:07:32.615 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-18T23:07:32.623 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-18T23:07:32.649 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-18T23:07:32.689 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T23:07:32.691 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T23:07:32.969 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-18T23:07:32.995 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-18T23:07:33.027 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T23:07:33.101 INFO:teuthology.orchestra.run.smithi142.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-2369-g1a6a3e74-1jammy [5,084 kB] 2024-04-18T23:07:33.126 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-18T23:07:33.153 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:33.201 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:33.356 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-18T23:07:33.369 INFO:teuthology.orchestra.run.smithi142.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-2369-g1a6a3e74-1jammy [244 kB] 2024-04-18T23:07:33.382 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-18T23:07:33.392 INFO:teuthology.orchestra.run.smithi142.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [124 kB] 2024-04-18T23:07:33.395 INFO:teuthology.orchestra.run.smithi142.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-2369-g1a6a3e74-1jammy [1,525 kB] 2024-04-18T23:07:33.422 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T23:07:33.498 INFO:teuthology.orchestra.run.smithi142.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-2369-g1a6a3e74-1jammy [6,246 kB] 2024-04-18T23:07:33.529 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-18T23:07:33.555 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:33.587 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:33.656 INFO:teuthology.orchestra.run.smithi120.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [14.0 kB] 2024-04-18T23:07:33.656 INFO:teuthology.orchestra.run.smithi120.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [869 kB] 2024-04-18T23:07:33.673 INFO:teuthology.orchestra.run.smithi120.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2369-g1a6a3e74-1jammy [2,435 kB] 2024-04-18T23:07:33.828 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-18T23:07:33.843 INFO:teuthology.orchestra.run.smithi120.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2369-g1a6a3e74-1jammy [791 kB] 2024-04-18T23:07:33.854 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:33.888 INFO:teuthology.orchestra.run.smithi142.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-2369-g1a6a3e74-1jammy [22.7 MB] 2024-04-18T23:07:33.894 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:33.906 INFO:teuthology.orchestra.run.smithi120.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2369-g1a6a3e74-1jammy [147 kB] 2024-04-18T23:07:33.909 INFO:teuthology.orchestra.run.smithi120.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2369-g1a6a3e74-1jammy [2,295 kB] 2024-04-18T23:07:33.924 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-18T23:07:33.950 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:33.991 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:34.066 INFO:teuthology.orchestra.run.smithi120.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2369-g1a6a3e74-1jammy [8,625 kB] 2024-04-18T23:07:34.298 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-18T23:07:34.324 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:34.364 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:34.377 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-18T23:07:34.403 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-18T23:07:34.444 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T23:07:34.692 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-18T23:07:34.706 INFO:teuthology.orchestra.run.smithi120.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2369-g1a6a3e74-1jammy [14.2 kB] 2024-04-18T23:07:34.706 INFO:teuthology.orchestra.run.smithi120.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2369-g1a6a3e74-1jammy [50.7 MB] 2024-04-18T23:07:34.718 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:34.730 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librados2. 2024-04-18T23:07:34.756 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../007-librados2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:34.759 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:34.788 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:35.062 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-18T23:07:35.088 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-18T23:07:35.137 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T23:07:35.233 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librbd1. 2024-04-18T23:07:35.259 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../008-librbd1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:35.299 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:35.301 INFO:teuthology.orchestra.run.smithi142.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-2369-g1a6a3e74-1jammy [14.0 kB] 2024-04-18T23:07:35.302 INFO:teuthology.orchestra.run.smithi142.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [869 kB] 2024-04-18T23:07:35.318 INFO:teuthology.orchestra.run.smithi142.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-2369-g1a6a3e74-1jammy [2,435 kB] 2024-04-18T23:07:35.427 INFO:teuthology.orchestra.run.smithi142.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-2369-g1a6a3e74-1jammy [791 kB] 2024-04-18T23:07:35.448 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librados2. 2024-04-18T23:07:35.473 INFO:teuthology.orchestra.run.smithi142.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-2369-g1a6a3e74-1jammy [147 kB] 2024-04-18T23:07:35.475 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../007-librados2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:35.478 INFO:teuthology.orchestra.run.smithi142.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-2369-g1a6a3e74-1jammy [2,295 kB] 2024-04-18T23:07:35.515 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:35.596 INFO:teuthology.orchestra.run.smithi142.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-2369-g1a6a3e74-1jammy [8,625 kB] 2024-04-18T23:07:35.727 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs2. 2024-04-18T23:07:35.753 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:35.794 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:35.976 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librbd1. 2024-04-18T23:07:36.002 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../008-librbd1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:36.051 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:36.071 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rados. 2024-04-18T23:07:36.077 INFO:teuthology.orchestra.run.smithi142.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-2369-g1a6a3e74-1jammy [14.2 kB] 2024-04-18T23:07:36.078 INFO:teuthology.orchestra.run.smithi142.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-2369-g1a6a3e74-1jammy [50.7 MB] 2024-04-18T23:07:36.086 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:36.121 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:36.496 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libcephfs2. 2024-04-18T23:07:36.522 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:36.554 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:36.632 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-18T23:07:36.659 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:36.699 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:36.815 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rados. 2024-04-18T23:07:36.830 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:36.873 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:36.951 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cephfs. 2024-04-18T23:07:36.967 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:37.010 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:37.193 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-18T23:07:37.220 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:37.296 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-18T23:07:37.322 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:37.354 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:37.393 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:37.646 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cephfs. 2024-04-18T23:07:37.661 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:37.666 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-18T23:07:37.693 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-18T23:07:37.704 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:37.724 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T23:07:37.982 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-18T23:07:38.008 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:38.040 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:38.043 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-prettytable. 2024-04-18T23:07:38.070 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-18T23:07:38.110 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-18T23:07:38.354 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rbd. 2024-04-18T23:07:38.360 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-18T23:07:38.381 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:38.386 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-18T23:07:38.435 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T23:07:38.440 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:38.724 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-18T23:07:38.746 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-prettytable. 2024-04-18T23:07:38.750 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-18T23:07:38.773 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-18T23:07:38.799 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T23:07:38.813 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-18T23:07:39.025 INFO:teuthology.orchestra.run.smithi120.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2369-g1a6a3e74-1jammy [121 kB] 2024-04-18T23:07:39.027 INFO:teuthology.orchestra.run.smithi120.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2369-g1a6a3e74-1jammy [38.9 kB] 2024-04-18T23:07:39.028 INFO:teuthology.orchestra.run.smithi120.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2369-g1a6a3e74-1jammy [13.4 MB] 2024-04-18T23:07:39.074 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rbd. 2024-04-18T23:07:39.076 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-18T23:07:39.089 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:39.092 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-18T23:07:39.101 INFO:teuthology.orchestra.run.smithi142.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-2369-g1a6a3e74-1jammy [121 kB] 2024-04-18T23:07:39.103 INFO:teuthology.orchestra.run.smithi142.stdout:Get:116 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-2369-g1a6a3e74-1jammy [38.9 kB] 2024-04-18T23:07:39.104 INFO:teuthology.orchestra.run.smithi142.stdout:Get:117 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-2369-g1a6a3e74-1jammy [13.4 MB] 2024-04-18T23:07:39.123 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:39.135 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T23:07:39.413 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-18T23:07:39.428 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-18T23:07:39.460 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-18T23:07:39.471 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T23:07:39.486 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-18T23:07:39.527 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T23:07:39.666 INFO:teuthology.orchestra.run.smithi142.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [91.4 kB] 2024-04-18T23:07:39.778 INFO:teuthology.orchestra.run.smithi120.stdout:Get:118 https://3.chacra.ceph.com/r/ceph/squid/1a6a3e742fd81c6d70be70552499bff767652c32/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-2369-g1a6a3e74-1jammy [91.4 kB] 2024-04-18T23:07:39.800 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua5.1. 2024-04-18T23:07:39.804 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-18T23:07:39.819 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-18T23:07:39.826 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-18T23:07:39.863 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T23:07:39.866 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-18T23:07:40.141 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-18T23:07:40.156 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-18T23:07:40.177 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua-any. 2024-04-18T23:07:40.190 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T23:07:40.193 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-18T23:07:40.196 INFO:teuthology.orchestra.run.smithi142.stdout:Fetched 179 MB in 21s (8,598 kB/s) 2024-04-18T23:07:40.236 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-18T23:07:40.339 INFO:teuthology.orchestra.run.smithi120.stdout:Fetched 179 MB in 21s (8,531 kB/s) 2024-04-18T23:07:40.353 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-18T23:07:40.432 INFO:teuthology.orchestra.run.smithi142.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-18T23:07:40.438 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-18T23:07:40.446 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package zip. 2024-04-18T23:07:40.462 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-18T23:07:40.468 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-04-18T23:07:40.496 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking zip (3.0-12build2) ... 2024-04-18T23:07:40.526 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T23:07:40.546 INFO:teuthology.orchestra.run.smithi120.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165995 files and directories currently installed.) 2024-04-18T23:07:40.552 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-04-18T23:07:40.553 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package lua5.1. 2024-04-18T23:07:40.579 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-18T23:07:40.619 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-18T23:07:40.625 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T23:07:40.808 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package unzip. 2024-04-18T23:07:40.822 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-18T23:07:40.835 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-18T23:07:40.848 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-18T23:07:40.875 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-18T23:07:40.888 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T23:07:40.931 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package lua-any. 2024-04-18T23:07:40.945 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-18T23:07:40.970 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-04-18T23:07:40.989 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-18T23:07:40.996 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-04-18T23:07:41.037 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T23:07:41.216 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package zip. 2024-04-18T23:07:41.229 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package luarocks. 2024-04-18T23:07:41.231 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-18T23:07:41.244 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-18T23:07:41.250 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-18T23:07:41.266 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-18T23:07:41.266 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking zip (3.0-12build2) ... 2024-04-18T23:07:41.278 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-18T23:07:41.308 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T23:07:41.398 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-04-18T23:07:41.424 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-04-18T23:07:41.464 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T23:07:41.562 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package unzip. 2024-04-18T23:07:41.588 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-18T23:07:41.620 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package librgw2. 2024-04-18T23:07:41.628 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-18T23:07:41.646 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../025-librgw2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:41.686 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:41.761 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-18T23:07:41.788 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:41.826 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-04-18T23:07:41.836 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:41.853 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:41.893 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:42.016 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package luarocks. 2024-04-18T23:07:42.042 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-18T23:07:42.082 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-18T23:07:42.214 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-18T23:07:42.238 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rgw. 2024-04-18T23:07:42.241 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:42.254 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:42.281 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:42.288 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:42.304 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-04-18T23:07:42.331 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:42.371 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:42.448 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package librgw2. 2024-04-18T23:07:42.475 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../025-librgw2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:42.515 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:42.600 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-18T23:07:42.617 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-18T23:07:42.626 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-18T23:07:42.643 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:42.667 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T23:07:42.684 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:42.715 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-04-18T23:07:42.742 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-04-18T23:07:42.782 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:07:42.979 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libradosstriper1. 2024-04-18T23:07:43.004 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:43.020 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-18T23:07:43.034 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rgw. 2024-04-18T23:07:43.035 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-18T23:07:43.044 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:43.060 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-04-18T23:07:43.060 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:43.070 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T23:07:43.075 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-04-18T23:07:43.101 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:43.110 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T23:07:43.356 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package librados2. 2024-04-18T23:07:43.371 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../007-librados2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:43.413 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package librados2. 2024-04-18T23:07:43.414 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:43.431 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-common. 2024-04-18T23:07:43.437 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-18T23:07:43.439 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../007-librados2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:43.458 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:43.464 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-18T23:07:43.479 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:43.497 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:43.504 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T23:07:43.815 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libradosstriper1. 2024-04-18T23:07:43.841 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:43.875 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package librbd1. 2024-04-18T23:07:43.881 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:43.902 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../008-librbd1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:43.942 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:43.973 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package librbd1. 2024-04-18T23:07:44.000 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../008-librbd1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:44.040 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:44.260 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-common. 2024-04-18T23:07:44.286 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:44.326 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:44.409 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libcephfs2. 2024-04-18T23:07:44.411 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libcephfs2. 2024-04-18T23:07:44.424 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:44.438 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:44.459 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:44.478 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:44.728 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-rados. 2024-04-18T23:07:44.730 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-rados. 2024-04-18T23:07:44.743 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:44.745 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../010-python3-rados_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:44.780 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:44.786 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:44.787 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-base. 2024-04-18T23:07:44.813 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:44.862 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:45.092 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-18T23:07:45.106 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-04-18T23:07:45.118 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:45.132 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:45.158 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:45.172 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:45.419 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-cephfs. 2024-04-18T23:07:45.434 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:45.441 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-18T23:07:45.467 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-18T23:07:45.477 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:45.507 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-18T23:07:45.566 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-cephfs. 2024-04-18T23:07:45.581 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:45.624 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:45.666 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-base. 2024-04-18T23:07:45.692 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:45.741 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:45.788 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-18T23:07:45.810 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cheroot. 2024-04-18T23:07:45.814 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:45.836 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-18T23:07:45.854 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:45.876 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T23:07:45.894 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-ceph-common. 2024-04-18T23:07:45.920 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:45.952 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:46.174 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-18T23:07:46.189 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-18T23:07:46.196 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-18T23:07:46.214 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-wcwidth. 2024-04-18T23:07:46.223 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-18T23:07:46.224 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T23:07:46.239 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-04-18T23:07:46.263 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-18T23:07:46.271 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T23:07:46.304 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-18T23:07:46.330 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-18T23:07:46.370 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-18T23:07:46.485 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-prettytable. 2024-04-18T23:07:46.512 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-18T23:07:46.544 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-18T23:07:46.574 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-18T23:07:46.591 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-prettytable. 2024-04-18T23:07:46.601 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-18T23:07:46.606 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-04-18T23:07:46.649 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-04-18T23:07:46.656 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cheroot. 2024-04-18T23:07:46.682 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-18T23:07:46.731 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T23:07:46.813 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-rbd. 2024-04-18T23:07:46.832 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-18T23:07:46.839 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:46.871 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:46.926 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-rbd. 2024-04-18T23:07:46.953 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:46.993 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:47.017 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-18T23:07:47.033 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-18T23:07:47.076 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-18T23:07:47.194 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-18T23:07:47.220 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-18T23:07:47.249 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-18T23:07:47.260 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-18T23:07:47.276 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-18T23:07:47.313 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-04-18T23:07:47.316 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T23:07:47.339 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-04-18T23:07:47.371 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T23:07:47.396 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-18T23:07:47.422 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-18T23:07:47.462 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-18T23:07:47.521 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-tempora. 2024-04-18T23:07:47.537 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-18T23:07:47.571 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-18T23:07:47.582 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-18T23:07:47.593 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-04-18T23:07:47.597 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-18T23:07:47.619 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-04-18T23:07:47.631 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T23:07:47.660 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T23:07:47.765 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-18T23:07:47.791 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-18T23:07:47.832 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-18T23:07:47.877 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-portend. 2024-04-18T23:07:47.903 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-18T23:07:47.934 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-18T23:07:47.943 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-18T23:07:47.960 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-18T23:07:47.962 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-04-18T23:07:47.978 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-04-18T23:07:48.001 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T23:07:48.013 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T23:07:48.135 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-tempora. 2024-04-18T23:07:48.150 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-18T23:07:48.185 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-18T23:07:48.221 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-18T23:07:48.247 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-18T23:07:48.287 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-18T23:07:48.316 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package lua5.1. 2024-04-18T23:07:48.321 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package lua5.1. 2024-04-18T23:07:48.342 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-18T23:07:48.347 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-04-18T23:07:48.382 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-18T23:07:48.387 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-04-18T23:07:48.463 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-portend. 2024-04-18T23:07:48.489 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-18T23:07:48.529 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-18T23:07:48.549 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-18T23:07:48.575 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-18T23:07:48.615 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-18T23:07:48.673 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package lua-any. 2024-04-18T23:07:48.686 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package lua-any. 2024-04-18T23:07:48.700 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-18T23:07:48.701 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-04-18T23:07:48.731 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-18T23:07:48.743 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking lua-any (27ubuntu1) ... 2024-04-18T23:07:48.832 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-18T23:07:48.858 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-18T23:07:48.899 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-18T23:07:48.951 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package zip. 2024-04-18T23:07:48.961 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-natsort. 2024-04-18T23:07:48.977 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-18T23:07:48.987 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-18T23:07:48.988 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package zip. 2024-04-18T23:07:49.014 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-04-18T23:07:49.017 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking zip (3.0-12build2) ... 2024-04-18T23:07:49.027 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-18T23:07:49.054 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking zip (3.0-12build2) ... 2024-04-18T23:07:49.202 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-18T23:07:49.228 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-18T23:07:49.268 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-18T23:07:49.280 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-logutils. 2024-04-18T23:07:49.295 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-18T23:07:49.330 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-18T23:07:49.337 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package unzip. 2024-04-18T23:07:49.341 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package unzip. 2024-04-18T23:07:49.363 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-18T23:07:49.367 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-04-18T23:07:49.399 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-18T23:07:49.403 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-04-18T23:07:49.600 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-18T23:07:49.614 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-natsort. 2024-04-18T23:07:49.626 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-18T23:07:49.640 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-18T23:07:49.666 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-18T23:07:49.680 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-18T23:07:49.795 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package luarocks. 2024-04-18T23:07:49.821 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-18T23:07:49.832 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package luarocks. 2024-04-18T23:07:49.858 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-04-18T23:07:49.861 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-18T23:07:49.899 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-04-18T23:07:49.992 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-logutils. 2024-04-18T23:07:50.018 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-18T23:07:50.058 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-18T23:07:50.252 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package librgw2. 2024-04-18T23:07:50.267 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package librgw2. 2024-04-18T23:07:50.269 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-mako. 2024-04-18T23:07:50.279 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../025-librgw2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:50.293 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../025-librgw2_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:50.297 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-18T23:07:50.319 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:50.350 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:50.370 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-18T23:07:50.396 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-18T23:07:50.436 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-18T23:07:50.661 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T23:07:50.739 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-mako. 2024-04-18T23:07:50.766 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-18T23:07:50.863 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-rgw. 2024-04-18T23:07:50.878 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:50.921 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:51.006 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-18T23:07:51.032 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-18T23:07:51.072 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-18T23:07:51.115 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T23:07:51.241 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-18T23:07:51.267 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-18T23:07:51.299 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T23:07:51.328 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-rgw. 2024-04-18T23:07:51.350 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-18T23:07:51.355 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:51.377 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-18T23:07:51.417 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-18T23:07:51.602 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libradosstriper1. 2024-04-18T23:07:51.628 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:51.668 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:51.670 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webob. 2024-04-18T23:07:51.696 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-18T23:07:51.728 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:51.736 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-18T23:07:51.851 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-18T23:07:51.878 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-18T23:07:51.918 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-18T23:07:52.057 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package liboath0:amd64. 2024-04-18T23:07:52.081 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-waitress. 2024-04-18T23:07:52.083 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-04-18T23:07:52.107 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-18T23:07:52.123 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T23:07:52.151 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T23:07:52.204 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-18T23:07:52.231 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-18T23:07:52.271 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-18T23:07:52.410 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libradosstriper1. 2024-04-18T23:07:52.436 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:52.459 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-tempita. 2024-04-18T23:07:52.468 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:52.486 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-18T23:07:52.534 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T23:07:52.549 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-webob. 2024-04-18T23:07:52.576 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-18T23:07:52.807 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-18T23:07:52.837 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-paste. 2024-04-18T23:07:52.863 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-18T23:07:52.872 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-common. 2024-04-18T23:07:52.888 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-common. 2024-04-18T23:07:52.899 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:52.903 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T23:07:52.915 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../029-ceph-common_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:52.938 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:52.963 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:53.135 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-waitress. 2024-04-18T23:07:53.161 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-18T23:07:53.204 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T23:07:53.223 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-18T23:07:53.250 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-18T23:07:53.290 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T23:07:53.513 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-tempita. 2024-04-18T23:07:53.539 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-18T23:07:53.580 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T23:07:53.601 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-18T23:07:53.628 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-18T23:07:53.668 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-18T23:07:53.883 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-paste. 2024-04-18T23:07:53.909 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-18T23:07:53.929 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-webtest. 2024-04-18T23:07:53.944 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-18T23:07:53.949 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T23:07:53.979 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-18T23:07:54.261 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-18T23:07:54.277 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-base. 2024-04-18T23:07:54.287 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-18T23:07:54.290 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pecan. 2024-04-18T23:07:54.294 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-base. 2024-04-18T23:07:54.304 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:54.317 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-18T23:07:54.320 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../030-ceph-base_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:54.327 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T23:07:54.353 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:54.357 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T23:07:54.370 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:54.630 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-18T23:07:54.657 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-18T23:07:54.697 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-18T23:07:54.703 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-certifi. 2024-04-18T23:07:54.730 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-18T23:07:54.770 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-18T23:07:54.940 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-18T23:07:54.950 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-webtest. 2024-04-18T23:07:54.965 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-18T23:07:54.966 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-04-18T23:07:54.966 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-18T23:07:54.992 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-04-18T23:07:55.000 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-18T23:07:55.006 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-18T23:07:55.032 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-04-18T23:07:55.065 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-idna. 2024-04-18T23:07:55.091 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-18T23:07:55.131 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-18T23:07:55.309 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-cheroot. 2024-04-18T23:07:55.318 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-cheroot. 2024-04-18T23:07:55.319 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pecan. 2024-04-18T23:07:55.336 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-18T23:07:55.345 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-04-18T23:07:55.345 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-18T23:07:55.375 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T23:07:55.385 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T23:07:55.394 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T23:07:55.426 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-urllib3. 2024-04-18T23:07:55.452 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-18T23:07:55.492 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T23:07:55.775 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-certifi. 2024-04-18T23:07:55.795 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-requests. 2024-04-18T23:07:55.801 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-18T23:07:55.822 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-18T23:07:55.841 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-18T23:07:55.862 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T23:07:55.970 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-18T23:07:55.971 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-04-18T23:07:55.985 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-18T23:07:55.998 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-04-18T23:07:56.020 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-18T23:07:56.038 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-04-18T23:07:56.161 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-idna. 2024-04-18T23:07:56.173 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-18T23:07:56.187 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-18T23:07:56.200 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-18T23:07:56.227 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-18T23:07:56.239 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T23:07:56.331 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-18T23:07:56.333 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jaraco.text. 2024-04-18T23:07:56.357 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-18T23:07:56.360 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-04-18T23:07:56.398 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-18T23:07:56.399 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-04-18T23:07:56.514 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-urllib3. 2024-04-18T23:07:56.518 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-18T23:07:56.534 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:56.540 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-18T23:07:56.568 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:56.580 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T23:07:56.702 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-18T23:07:56.709 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-04-18T23:07:56.718 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-18T23:07:56.735 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-04-18T23:07:56.752 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-18T23:07:56.775 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-04-18T23:07:56.892 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-requests. 2024-04-18T23:07:56.918 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-18T23:07:56.931 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-18T23:07:56.950 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T23:07:56.958 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:56.997 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:57.031 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-tempora. 2024-04-18T23:07:57.057 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-18T23:07:57.078 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-tempora. 2024-04-18T23:07:57.097 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-18T23:07:57.104 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-04-18T23:07:57.144 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-04-18T23:07:57.203 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-18T23:07:57.218 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-18T23:07:57.261 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T23:07:57.275 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-18T23:07:57.302 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-18T23:07:57.333 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-18T23:07:57.372 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-portend. 2024-04-18T23:07:57.399 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-18T23:07:57.400 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-portend. 2024-04-18T23:07:57.427 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-04-18T23:07:57.439 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-18T23:07:57.467 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-04-18T23:07:57.531 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-18T23:07:57.558 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:07:57.586 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-openssl. 2024-04-18T23:07:57.589 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:57.613 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-18T23:07:57.653 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-18T23:07:57.742 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-18T23:07:57.745 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-04-18T23:07:57.768 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-18T23:07:57.772 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-04-18T23:07:57.808 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-18T23:07:57.811 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-04-18T23:07:57.940 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr. 2024-04-18T23:07:57.961 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-18T23:07:57.967 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:57.988 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:57.998 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:58.028 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:58.111 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-18T23:07:58.123 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-cherrypy3. 2024-04-18T23:07:58.137 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-18T23:07:58.150 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-04-18T23:07:58.178 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-18T23:07:58.190 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-04-18T23:07:58.314 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-18T23:07:58.341 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-18T23:07:58.376 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mon. 2024-04-18T23:07:58.381 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-18T23:07:58.403 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:58.442 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:58.531 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-natsort. 2024-04-18T23:07:58.535 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-natsort. 2024-04-18T23:07:58.557 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-18T23:07:58.562 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-04-18T23:07:58.597 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-18T23:07:58.602 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-04-18T23:07:58.642 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-openssl. 2024-04-18T23:07:58.668 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-18T23:07:58.708 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-18T23:07:58.872 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-logutils. 2024-04-18T23:07:58.884 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-logutils. 2024-04-18T23:07:58.899 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-18T23:07:58.911 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-04-18T23:07:58.939 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-18T23:07:58.942 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-04-18T23:07:59.045 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr. 2024-04-18T23:07:59.071 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-osd. 2024-04-18T23:07:59.071 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:59.098 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:59.111 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:59.137 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:59.212 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-18T23:07:59.238 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-18T23:07:59.242 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-markupsafe. 2024-04-18T23:07:59.269 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-04-18T23:07:59.278 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-18T23:07:59.308 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-04-18T23:07:59.532 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mon. 2024-04-18T23:07:59.558 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:07:59.598 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-mako. 2024-04-18T23:07:59.598 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:07:59.611 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-mako. 2024-04-18T23:07:59.624 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-18T23:07:59.638 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-04-18T23:07:59.664 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T23:07:59.678 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T23:08:00.010 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-18T23:08:00.015 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-simplegeneric. 2024-04-18T23:08:00.028 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-18T23:08:00.036 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-04-18T23:08:00.073 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-18T23:08:00.076 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-04-18T23:08:00.135 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-osd. 2024-04-18T23:08:00.161 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:00.199 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph. 2024-04-18T23:08:00.201 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:00.215 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../066-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:00.249 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:00.320 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-18T23:08:00.347 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-18T23:08:00.359 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-singledispatch. 2024-04-18T23:08:00.386 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-04-18T23:08:00.387 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-18T23:08:00.418 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-04-18T23:08:00.519 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-fuse. 2024-04-18T23:08:00.546 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:00.585 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:00.679 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-webob. 2024-04-18T23:08:00.689 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-webob. 2024-04-18T23:08:00.694 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-18T23:08:00.715 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-04-18T23:08:00.737 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-18T23:08:00.756 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-04-18T23:08:00.963 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mds. 2024-04-18T23:08:00.990 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:01.030 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:01.116 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-waitress. 2024-04-18T23:08:01.143 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-18T23:08:01.186 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T23:08:01.217 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-waitress. 2024-04-18T23:08:01.244 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-04-18T23:08:01.255 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph. 2024-04-18T23:08:01.281 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../066-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:01.287 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T23:08:01.321 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:01.424 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package cephadm. 2024-04-18T23:08:01.451 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../069-cephadm_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:01.477 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-tempita. 2024-04-18T23:08:01.490 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:01.504 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-18T23:08:01.544 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T23:08:01.595 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-tempita. 2024-04-18T23:08:01.622 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-04-18T23:08:01.641 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-fuse. 2024-04-18T23:08:01.662 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T23:08:01.668 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:01.707 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:01.810 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-18T23:08:01.813 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-paste. 2024-04-18T23:08:01.837 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-18T23:08:01.840 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-18T23:08:01.871 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T23:08:01.877 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-18T23:08:01.940 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-paste. 2024-04-18T23:08:01.966 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-04-18T23:08:01.998 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T23:08:02.036 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mds. 2024-04-18T23:08:02.062 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:02.094 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:02.180 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-18T23:08:02.192 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-18T23:08:02.207 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T23:08:02.208 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-18T23:08:02.250 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T23:08:02.317 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-04-18T23:08:02.333 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-04-18T23:08:02.376 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T23:08:02.379 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:02.489 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package cephadm. 2024-04-18T23:08:02.511 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-18T23:08:02.515 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../069-cephadm_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:02.527 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-18T23:08:02.555 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:02.561 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-18T23:08:02.645 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pastedeploy. 2024-04-18T23:08:02.660 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-04-18T23:08:02.695 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-04-18T23:08:02.865 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-webtest. 2024-04-18T23:08:02.875 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-18T23:08:02.891 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-18T23:08:02.901 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-18T23:08:02.931 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-18T23:08:02.941 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-18T23:08:02.998 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-webtest. 2024-04-18T23:08:03.024 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-04-18T23:08:03.064 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-04-18T23:08:03.243 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pecan. 2024-04-18T23:08:03.261 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-18T23:08:03.269 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-18T23:08:03.287 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T23:08:03.309 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T23:08:03.347 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:03.376 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pecan. 2024-04-18T23:08:03.402 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-04-18T23:08:03.409 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-babel. 2024-04-18T23:08:03.437 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T23:08:03.442 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T23:08:03.484 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:03.698 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-certifi. 2024-04-18T23:08:03.725 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-18T23:08:03.764 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-18T23:08:03.804 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-jinja2. 2024-04-18T23:08:03.822 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-certifi. 2024-04-18T23:08:03.831 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-18T23:08:03.848 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-04-18T23:08:03.870 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T23:08:03.889 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-04-18T23:08:04.059 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-idna. 2024-04-18T23:08:04.074 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-18T23:08:04.109 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-18T23:08:04.192 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-idna. 2024-04-18T23:08:04.218 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-04-18T23:08:04.223 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-18T23:08:04.251 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:04.258 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-idna (3.3-1) ... 2024-04-18T23:08:04.290 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:04.308 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-babel. 2024-04-18T23:08:04.335 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T23:08:04.366 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:04.421 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-urllib3. 2024-04-18T23:08:04.447 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-18T23:08:04.487 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T23:08:04.511 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-urllib3. 2024-04-18T23:08:04.537 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-04-18T23:08:04.577 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T23:08:04.635 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-18T23:08:04.662 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-18T23:08:04.685 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-jinja2. 2024-04-18T23:08:04.701 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-18T23:08:04.712 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-18T23:08:04.752 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T23:08:04.816 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-requests. 2024-04-18T23:08:04.842 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-18T23:08:04.872 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-requests. 2024-04-18T23:08:04.882 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T23:08:04.899 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-04-18T23:08:04.938 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T23:08:04.979 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-routes. 2024-04-18T23:08:04.995 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-18T23:08:05.029 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T23:08:05.114 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-18T23:08:05.141 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:05.180 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:05.194 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-18T23:08:05.220 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-18T23:08:05.258 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-werkzeug. 2024-04-18T23:08:05.260 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T23:08:05.285 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-04-18T23:08:05.325 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T23:08:05.357 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-18T23:08:05.373 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:05.415 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:05.525 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-18T23:08:05.530 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-18T23:08:05.546 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:05.552 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-18T23:08:05.589 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:05.592 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-18T23:08:05.611 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-04-18T23:08:05.638 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:05.678 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:05.895 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-routes. 2024-04-18T23:08:05.923 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-18T23:08:05.961 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T23:08:05.987 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-18T23:08:06.014 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:06.041 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-04-18T23:08:06.068 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:06.108 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:06.111 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:06.298 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-18T23:08:06.325 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:06.364 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:06.410 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-18T23:08:06.422 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-bcrypt. 2024-04-18T23:08:06.437 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-18T23:08:06.449 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-04-18T23:08:06.477 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-18T23:08:06.505 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-04-18T23:08:06.629 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-18T23:08:06.657 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-18T23:08:06.695 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T23:08:06.780 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-openssl. 2024-04-18T23:08:06.783 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-openssl. 2024-04-18T23:08:06.806 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-18T23:08:06.807 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-04-18T23:08:06.842 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-18T23:08:06.886 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-04-18T23:08:07.154 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr. 2024-04-18T23:08:07.180 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:07.191 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr. 2024-04-18T23:08:07.216 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-joblib. 2024-04-18T23:08:07.217 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:07.220 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:07.243 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-18T23:08:07.266 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:07.282 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T23:08:07.599 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mon. 2024-04-18T23:08:07.611 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-18T23:08:07.625 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:07.627 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-18T23:08:07.636 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mon. 2024-04-18T23:08:07.653 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-18T23:08:07.662 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:07.665 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:07.670 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T23:08:07.680 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-18T23:08:07.693 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:07.719 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-18T23:08:08.014 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-sklearn. 2024-04-18T23:08:08.041 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-18T23:08:08.080 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T23:08:08.219 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-osd. 2024-04-18T23:08:08.245 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:08.285 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:08.356 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-osd. 2024-04-18T23:08:08.382 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:08.390 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-joblib. 2024-04-18T23:08:08.417 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-18T23:08:08.422 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:08.456 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T23:08:08.739 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-18T23:08:08.767 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:08.810 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-18T23:08:08.837 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-18T23:08:08.870 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-18T23:08:08.882 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:09.155 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-sklearn. 2024-04-18T23:08:09.182 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-18T23:08:09.221 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T23:08:09.347 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph. 2024-04-18T23:08:09.373 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../066-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:09.413 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:09.467 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph. 2024-04-18T23:08:09.482 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../066-ceph_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:09.525 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:09.742 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-fuse. 2024-04-18T23:08:09.768 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:09.802 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-fuse. 2024-04-18T23:08:09.808 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:09.826 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:09.844 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-cachetools. 2024-04-18T23:08:09.861 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:09.872 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-18T23:08:09.911 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-18T23:08:09.964 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-18T23:08:09.991 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:10.116 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:10.145 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mds. 2024-04-18T23:08:10.172 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:10.197 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-18T23:08:10.203 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:10.206 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mds. 2024-04-18T23:08:10.224 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-18T23:08:10.232 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:10.263 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-18T23:08:10.272 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:10.557 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package cephadm. 2024-04-18T23:08:10.583 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../069-cephadm_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:10.600 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-18T23:08:10.623 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:10.628 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-18T23:08:10.667 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T23:08:10.677 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package cephadm. 2024-04-18T23:08:10.704 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../069-cephadm_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:10.743 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:10.943 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-18T23:08:10.962 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-rsa. 2024-04-18T23:08:10.969 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-18T23:08:10.989 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-18T23:08:11.004 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-asyncssh. 2024-04-18T23:08:11.009 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-18T23:08:11.028 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-18T23:08:11.031 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-04-18T23:08:11.062 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-04-18T23:08:11.186 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-cachetools. 2024-04-18T23:08:11.213 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-18T23:08:11.252 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-18T23:08:11.349 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-google-auth. 2024-04-18T23:08:11.376 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-18T23:08:11.415 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-18T23:08:11.445 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-18T23:08:11.472 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T23:08:11.490 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python-babel-localedata. 2024-04-18T23:08:11.506 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T23:08:11.512 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:11.548 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:11.557 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-18T23:08:11.584 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-18T23:08:11.615 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-18T23:08:11.710 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-18T23:08:11.726 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-18T23:08:11.760 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T23:08:11.886 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-18T23:08:11.912 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-18T23:08:11.952 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T23:08:12.021 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-websocket. 2024-04-18T23:08:12.049 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-18T23:08:12.088 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-18T23:08:12.221 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-rsa. 2024-04-18T23:08:12.249 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-18T23:08:12.288 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-18T23:08:12.416 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-18T23:08:12.443 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-18T23:08:12.482 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T23:08:12.511 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-babel. 2024-04-18T23:08:12.527 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T23:08:12.543 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-babel. 2024-04-18T23:08:12.561 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:12.570 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-04-18T23:08:12.601 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:12.608 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-google-auth. 2024-04-18T23:08:12.635 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-18T23:08:12.675 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-18T23:08:12.856 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-jinja2. 2024-04-18T23:08:12.879 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-jinja2. 2024-04-18T23:08:12.883 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-18T23:08:12.906 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-04-18T23:08:12.922 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T23:08:12.945 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T23:08:12.995 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-18T23:08:13.022 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-18T23:08:13.061 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T23:08:13.105 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-18T23:08:13.121 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:13.163 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:13.242 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-18T23:08:13.257 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-04-18T23:08:13.265 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:13.284 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:13.309 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:13.324 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:13.372 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-websocket. 2024-04-18T23:08:13.399 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-18T23:08:13.439 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-18T23:08:13.483 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-18T23:08:13.510 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-18T23:08:13.549 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T23:08:13.604 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-18T23:08:13.627 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-repoze.lru. 2024-04-18T23:08:13.631 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-18T23:08:13.654 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-04-18T23:08:13.670 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-18T23:08:13.694 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-04-18T23:08:13.759 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-18T23:08:13.775 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-18T23:08:13.809 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T23:08:13.861 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-18T23:08:13.888 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T23:08:13.927 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T23:08:13.973 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-routes. 2024-04-18T23:08:13.997 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-routes. 2024-04-18T23:08:14.000 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-18T23:08:14.023 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-04-18T23:08:14.039 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T23:08:14.063 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T23:08:14.155 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package jq. 2024-04-18T23:08:14.170 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T23:08:14.213 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-18T23:08:14.376 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-18T23:08:14.403 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:14.417 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-04-18T23:08:14.431 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-18T23:08:14.442 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:14.444 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:14.458 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:14.465 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package socat. 2024-04-18T23:08:14.481 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-18T23:08:14.484 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:14.497 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:14.524 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-18T23:08:14.826 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-18T23:08:14.836 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package xmlstarlet. 2024-04-18T23:08:14.851 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-18T23:08:14.852 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-18T23:08:14.894 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-18T23:08:14.908 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T23:08:15.240 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-test. 2024-04-18T23:08:15.253 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-18T23:08:15.255 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:15.280 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T23:08:15.311 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T23:08:15.331 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:15.555 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package jq. 2024-04-18T23:08:15.571 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T23:08:15.613 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-18T23:08:15.639 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-18T23:08:15.666 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-18T23:08:15.706 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T23:08:15.714 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-04-18T23:08:15.741 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-04-18T23:08:15.781 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T23:08:15.875 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package socat. 2024-04-18T23:08:15.890 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-18T23:08:15.933 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-18T23:08:16.226 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-joblib. 2024-04-18T23:08:16.238 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package xmlstarlet. 2024-04-18T23:08:16.253 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-18T23:08:16.260 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-18T23:08:16.276 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-joblib. 2024-04-18T23:08:16.292 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T23:08:16.302 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-18T23:08:16.303 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-04-18T23:08:16.342 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T23:08:16.604 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-18T23:08:16.619 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-18T23:08:16.648 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-test. 2024-04-18T23:08:16.653 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-18T23:08:16.663 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-04-18T23:08:16.664 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:16.690 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-04-18T23:08:16.698 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:16.721 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-04-18T23:08:16.965 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-sklearn. 2024-04-18T23:08:16.992 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-18T23:08:17.008 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-sklearn. 2024-04-18T23:08:17.031 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T23:08:17.034 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-04-18T23:08:17.074 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T23:08:17.661 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package ceph-volume. 2024-04-18T23:08:17.688 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:17.714 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-18T23:08:17.727 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:17.742 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:17.800 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-04-18T23:08:17.828 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:17.900 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:17.952 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:18.264 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-18T23:08:18.281 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:18.314 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:18.634 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-18T23:08:18.661 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-18T23:08:18.700 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T23:08:18.845 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-cachetools. 2024-04-18T23:08:18.872 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-18T23:08:18.911 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-18T23:08:19.006 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-cachetools. 2024-04-18T23:08:19.013 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-18T23:08:19.033 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-04-18T23:08:19.041 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-18T23:08:19.073 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-04-18T23:08:19.079 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-18T23:08:19.086 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package ceph-volume. 2024-04-18T23:08:19.114 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:19.152 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:19.156 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-18T23:08:19.172 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-18T23:08:19.214 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-18T23:08:19.351 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pyasn1. 2024-04-18T23:08:19.378 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-04-18T23:08:19.418 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-04-18T23:08:19.425 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package nvme-cli. 2024-04-18T23:08:19.431 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-18T23:08:19.447 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:19.453 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-18T23:08:19.483 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T23:08:19.489 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:19.542 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-18T23:08:19.570 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-18T23:08:19.609 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T23:08:19.713 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-04-18T23:08:19.740 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-04-18T23:08:19.775 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-18T23:08:19.779 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T23:08:19.803 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-18T23:08:19.842 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T23:08:19.905 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package pkg-config. 2024-04-18T23:08:19.929 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-rsa. 2024-04-18T23:08:19.933 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-18T23:08:19.956 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-18T23:08:19.971 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T23:08:19.995 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-18T23:08:20.066 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-rsa. 2024-04-18T23:08:20.093 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-04-18T23:08:20.133 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-rsa (4.8-1) ... 2024-04-18T23:08:20.146 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-18T23:08:20.173 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-18T23:08:20.213 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-18T23:08:20.233 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-18T23:08:20.261 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-18T23:08:20.299 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-18T23:08:20.307 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-google-auth. 2024-04-18T23:08:20.334 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-18T23:08:20.373 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-18T23:08:20.403 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-google-auth. 2024-04-18T23:08:20.427 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-04-18T23:08:20.461 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-04-18T23:08:20.550 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package nvme-cli. 2024-04-18T23:08:20.577 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-18T23:08:20.617 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T23:08:20.702 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-18T23:08:20.728 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-18T23:08:20.729 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-18T23:08:20.755 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-18T23:08:20.768 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T23:08:20.794 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-18T23:08:20.798 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-04-18T23:08:20.825 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-04-18T23:08:20.865 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T23:08:21.047 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package pkg-config. 2024-04-18T23:08:21.072 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pastescript. 2024-04-18T23:08:21.074 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-18T23:08:21.079 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-websocket. 2024-04-18T23:08:21.088 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-18T23:08:21.106 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-18T23:08:21.113 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T23:08:21.121 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-18T23:08:21.146 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-18T23:08:21.177 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-websocket. 2024-04-18T23:08:21.203 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-04-18T23:08:21.243 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-04-18T23:08:21.358 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-18T23:08:21.373 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-18T23:08:21.392 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pluggy. 2024-04-18T23:08:21.410 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-18T23:08:21.419 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-18T23:08:21.458 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-18T23:08:21.466 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-18T23:08:21.493 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-18T23:08:21.532 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T23:08:21.737 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-psutil. 2024-04-18T23:08:21.764 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-18T23:08:21.772 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-kubernetes. 2024-04-18T23:08:21.794 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-18T23:08:21.794 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-18T23:08:21.798 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-04-18T23:08:21.821 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-18T23:08:21.838 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T23:08:21.852 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-18T23:08:22.130 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pastescript. 2024-04-18T23:08:22.148 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-py. 2024-04-18T23:08:22.157 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-18T23:08:22.175 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-18T23:08:22.179 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-18T23:08:22.207 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:22.213 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-18T23:08:22.222 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-18T23:08:22.246 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:22.453 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-04-18T23:08:22.469 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:22.502 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:22.534 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pygments. 2024-04-18T23:08:22.557 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-18T23:08:22.562 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-18T23:08:22.566 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pluggy. 2024-04-18T23:08:22.584 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-18T23:08:22.594 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-18T23:08:22.609 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T23:08:22.623 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T23:08:22.633 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-18T23:08:22.806 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libonig5:amd64. 2024-04-18T23:08:22.833 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-04-18T23:08:22.872 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T23:08:22.926 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-18T23:08:22.944 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-psutil. 2024-04-18T23:08:22.953 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T23:08:22.971 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-18T23:08:22.984 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T23:08:23.011 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-18T23:08:23.046 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-18T23:08:23.074 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-18T23:08:23.113 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-18T23:08:23.226 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libjq1:amd64. 2024-04-18T23:08:23.253 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T23:08:23.254 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package jq. 2024-04-18T23:08:23.270 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T23:08:23.292 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T23:08:23.312 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-18T23:08:23.355 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-py. 2024-04-18T23:08:23.382 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-18T23:08:23.391 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-toml. 2024-04-18T23:08:23.418 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-18T23:08:23.422 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-18T23:08:23.457 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-18T23:08:23.536 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package jq. 2024-04-18T23:08:23.552 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-04-18T23:08:23.564 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package socat. 2024-04-18T23:08:23.580 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-18T23:08:23.586 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-04-18T23:08:23.623 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-18T23:08:23.709 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pygments. 2024-04-18T23:08:23.736 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-18T23:08:23.775 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T23:08:23.785 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-pytest. 2024-04-18T23:08:23.813 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-18T23:08:23.814 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package socat. 2024-04-18T23:08:23.841 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-04-18T23:08:23.851 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T23:08:23.880 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-04-18T23:08:23.951 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package xmlstarlet. 2024-04-18T23:08:23.967 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-18T23:08:24.009 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-18T23:08:24.172 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package python3-simplejson. 2024-04-18T23:08:24.176 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package xmlstarlet. 2024-04-18T23:08:24.200 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-18T23:08:24.203 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-04-18T23:08:24.212 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-18T23:08:24.238 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-18T23:08:24.240 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-18T23:08:24.242 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-04-18T23:08:24.278 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-18T23:08:24.364 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-test. 2024-04-18T23:08:24.391 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:24.430 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:24.499 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-18T23:08:24.527 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-18T23:08:24.566 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-18T23:08:24.581 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-toml. 2024-04-18T23:08:24.606 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-test. 2024-04-18T23:08:24.609 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-18T23:08:24.633 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../097-ceph-test_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:24.647 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-18T23:08:24.672 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:24.951 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-pytest. 2024-04-18T23:08:24.978 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-18T23:08:25.017 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T23:08:25.261 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package radosgw. 2024-04-18T23:08:25.290 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../116-radosgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:25.328 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:25.337 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package python3-simplejson. 2024-04-18T23:08:25.364 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-18T23:08:25.403 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-18T23:08:25.681 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-18T23:08:25.708 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-18T23:08:25.747 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-18T23:08:26.214 INFO:teuthology.orchestra.run.smithi136.stdout:Selecting previously unselected package rbd-fuse. 2024-04-18T23:08:26.230 INFO:teuthology.orchestra.run.smithi136.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:26.272 INFO:teuthology.orchestra.run.smithi136.stdout:Unpacking rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:26.518 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package radosgw. 2024-04-18T23:08:26.546 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../116-radosgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:26.568 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-18T23:08:26.584 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:26.709 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package ceph-volume. 2024-04-18T23:08:26.737 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:26.776 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:26.819 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T23:08:26.945 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T23:08:26.960 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package ceph-volume. 2024-04-18T23:08:26.987 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-2369-g1a6a3e74-1jammy_all.deb ... 2024-04-18T23:08:27.018 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:27.096 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-18T23:08:27.124 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:27.163 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:27.205 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-18T23:08:27.289 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package libcephfs-dev. 2024-04-18T23:08:27.305 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:27.339 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:27.370 INFO:teuthology.orchestra.run.smithi073.stdout:Selecting previously unselected package rbd-fuse. 2024-04-18T23:08:27.387 INFO:teuthology.orchestra.run.smithi073.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:27.429 INFO:teuthology.orchestra.run.smithi073.stdout:Unpacking rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:27.457 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-18T23:08:27.473 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-18T23:08:27.507 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T23:08:27.541 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-18T23:08:27.642 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package lua-socket:amd64. 2024-04-18T23:08:27.669 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-04-18T23:08:27.709 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T23:08:27.725 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-18T23:08:27.778 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-18T23:08:27.794 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-18T23:08:27.828 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-18T23:08:27.975 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T23:08:27.980 INFO:teuthology.orchestra.run.smithi136.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-18T23:08:28.030 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package lua-sec:amd64. 2024-04-18T23:08:28.047 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:28.058 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-04-18T23:08:28.097 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-04-18T23:08:28.102 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T23:08:28.174 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package nvme-cli. 2024-04-18T23:08:28.201 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-18T23:08:28.240 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T23:08:28.365 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-18T23:08:28.401 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package nvme-cli. 2024-04-18T23:08:28.429 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-04-18T23:08:28.468 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T23:08:28.548 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user cephadm....done 2024-04-18T23:08:28.654 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package pkg-config. 2024-04-18T23:08:28.681 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-18T23:08:28.688 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T23:08:28.699 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-18T23:08:28.720 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T23:08:28.865 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package pkg-config. 2024-04-18T23:08:28.892 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-04-18T23:08:28.931 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T23:08:28.973 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-18T23:08:28.981 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-18T23:08:28.997 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-18T23:08:29.039 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-18T23:08:29.147 INFO:teuthology.orchestra.run.smithi073.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-18T23:08:29.220 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-04-18T23:08:29.220 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:29.224 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-18T23:08:29.236 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-04-18T23:08:29.278 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-04-18T23:08:29.333 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-18T23:08:29.467 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-18T23:08:29.495 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-18T23:08:29.534 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-18T23:08:29.567 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-18T23:08:29.690 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-iniconfig. 2024-04-18T23:08:29.717 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-04-18T23:08:29.744 INFO:teuthology.orchestra.run.smithi073.stdout:Adding system user cephadm....done 2024-04-18T23:08:29.756 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-04-18T23:08:29.803 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pastescript. 2024-04-18T23:08:29.819 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-18T23:08:29.827 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T23:08:29.853 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-18T23:08:29.892 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T23:08:29.953 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-18T23:08:30.018 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pastescript. 2024-04-18T23:08:30.045 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-04-18T23:08:30.075 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-04-18T23:08:30.131 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pluggy. 2024-04-18T23:08:30.159 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-18T23:08:30.185 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-18T23:08:30.198 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-18T23:08:30.254 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T23:08:30.379 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pluggy. 2024-04-18T23:08:30.407 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-04-18T23:08:30.446 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-04-18T23:08:30.453 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-18T23:08:30.509 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-psutil. 2024-04-18T23:08:30.525 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-18T23:08:30.563 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-18T23:08:30.567 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-18T23:08:30.623 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-18T23:08:30.758 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-psutil. 2024-04-18T23:08:30.785 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-04-18T23:08:30.797 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-18T23:08:30.824 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-04-18T23:08:30.874 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-18T23:08:30.895 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-py. 2024-04-18T23:08:30.923 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-18T23:08:30.961 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-18T23:08:31.032 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T23:08:31.125 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-18T23:08:31.158 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-18T23:08:31.169 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-py. 2024-04-18T23:08:31.196 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-04-18T23:08:31.235 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-py (1.10.0-1) ... 2024-04-18T23:08:31.290 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pygments. 2024-04-18T23:08:31.317 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-18T23:08:31.356 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T23:08:31.376 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-18T23:08:31.459 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T23:08:31.556 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pygments. 2024-04-18T23:08:31.583 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-04-18T23:08:31.622 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T23:08:31.653 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-18T23:08:31.811 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-18T23:08:31.954 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-18T23:08:32.054 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-18T23:08:32.101 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-18T23:08:32.129 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-18T23:08:32.135 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pyinotify. 2024-04-18T23:08:32.151 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-04-18T23:08:32.168 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-18T23:08:32.193 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-04-18T23:08:32.213 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:32.313 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-18T23:08:32.455 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-toml. 2024-04-18T23:08:32.463 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-toml. 2024-04-18T23:08:32.480 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T23:08:32.482 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-18T23:08:32.490 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-04-18T23:08:32.512 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-18T23:08:32.524 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-18T23:08:32.529 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-04-18T23:08:32.606 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T23:08:32.715 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T23:08:32.774 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-pytest. 2024-04-18T23:08:32.775 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-18T23:08:32.801 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-18T23:08:32.816 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-18T23:08:32.832 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-pytest. 2024-04-18T23:08:32.840 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T23:08:32.860 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-04-18T23:08:32.898 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T23:08:33.034 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-18T23:08:33.067 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-18T23:08:33.185 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package python3-simplejson. 2024-04-18T23:08:33.213 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-18T23:08:33.244 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package python3-simplejson. 2024-04-18T23:08:33.252 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-18T23:08:33.260 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-04-18T23:08:33.293 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:33.302 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-04-18T23:08:33.452 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-18T23:08:33.496 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-18T23:08:33.512 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-18T23:08:33.554 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-18T23:08:33.555 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-04-18T23:08:33.567 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-18T23:08:33.571 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-04-18T23:08:33.613 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-04-18T23:08:33.617 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T23:08:33.787 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T23:08:33.821 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T23:08:33.913 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T23:08:33.946 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-18T23:08:34.022 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-18T23:08:34.283 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package radosgw. 2024-04-18T23:08:34.290 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-18T23:08:34.311 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../116-radosgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:34.314 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:34.349 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:34.360 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package radosgw. 2024-04-18T23:08:34.388 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../116-radosgw_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:34.425 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-18T23:08:34.425 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:34.666 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T23:08:34.934 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-18T23:08:34.956 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-18T23:08:34.988 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-18T23:08:35.051 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T23:08:35.161 INFO:teuthology.orchestra.run.smithi142.stdout:Selecting previously unselected package rbd-fuse. 2024-04-18T23:08:35.178 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-18T23:08:35.189 INFO:teuthology.orchestra.run.smithi142.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:35.196 INFO:teuthology.orchestra.run.smithi120.stdout:Selecting previously unselected package rbd-fuse. 2024-04-18T23:08:35.212 INFO:teuthology.orchestra.run.smithi120.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-2369-g1a6a3e74-1jammy_amd64.deb ... 2024-04-18T23:08:35.236 INFO:teuthology.orchestra.run.smithi142.stdout:Unpacking rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:35.244 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up zip (3.0-12build2) ... 2024-04-18T23:08:35.246 INFO:teuthology.orchestra.run.smithi120.stdout:Unpacking rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:35.369 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T23:08:35.523 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-18T23:08:35.537 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:35.567 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-04-18T23:08:35.663 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-18T23:08:35.774 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T23:08:35.801 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-04-18T23:08:35.908 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T23:08:35.914 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T23:08:35.936 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-04-18T23:08:36.063 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T23:08:36.157 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-18T23:08:36.178 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-18T23:08:36.209 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-04-18T23:08:36.322 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T23:08:36.440 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-18T23:08:36.466 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up zip (3.0-12build2) ... 2024-04-18T23:08:36.524 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-18T23:08:36.557 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T23:08:36.565 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-04-18T23:08:36.592 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T23:08:36.917 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-18T23:08:36.961 INFO:teuthology.orchestra.run.smithi142.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-18T23:08:36.978 INFO:teuthology.orchestra.run.smithi120.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-04-18T23:08:37.035 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:37.255 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:37.295 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T23:08:37.309 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-18T23:08:37.510 INFO:teuthology.orchestra.run.smithi142.stdout:Adding system user cephadm....done 2024-04-18T23:08:37.537 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T23:08:37.569 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T23:08:37.663 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-18T23:08:37.666 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T23:08:37.772 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T23:08:37.822 INFO:teuthology.orchestra.run.smithi120.stdout:Adding system user cephadm....done 2024-04-18T23:08:37.951 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-18T23:08:37.962 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T23:08:37.978 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-04-18T23:08:38.056 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-18T23:08:38.210 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-18T23:08:38.255 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-04-18T23:08:38.263 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T23:08:38.311 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-18T23:08:38.433 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-18T23:08:38.498 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-04-18T23:08:38.546 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-18T23:08:38.606 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-18T23:08:38.607 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-04-18T23:08:38.684 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T23:08:38.788 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T23:08:38.842 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-04-18T23:08:38.865 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-18T23:08:38.914 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-18T23:08:38.975 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:39.044 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T23:08:39.093 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-04-18T23:08:39.216 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T23:08:39.220 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-py (1.10.0-1) ... 2024-04-18T23:08:39.259 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-idna (3.3-1) ... 2024-04-18T23:08:39.328 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T23:08:39.513 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-04-18T23:08:39.577 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T23:08:39.635 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-18T23:08:39.671 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-18T23:08:39.848 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-04-18T23:08:39.902 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-18T23:08:39.914 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-18T23:08:40.023 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:40.092 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-04-18T23:08:40.154 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-18T23:08:40.316 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-idna (3.3-1) ... 2024-04-18T23:08:40.368 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-04-18T23:08:40.405 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-18T23:08:40.628 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-04-18T23:08:40.642 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T23:08:40.656 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-18T23:08:40.740 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T23:08:40.871 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-04-18T23:08:40.915 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-18T23:08:40.967 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:41.110 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-18T23:08:41.122 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-04-18T23:08:41.175 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:41.369 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T23:08:41.414 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-ceph-argparse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:41.484 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T23:08:41.495 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-18T23:08:41.619 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T23:08:41.621 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T23:08:41.657 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-04-18T23:08:41.736 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T23:08:41.783 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-04-18T23:08:41.806 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T23:08:41.837 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-18T23:08:41.914 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-18T23:08:41.918 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-04-18T23:08:42.044 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-04-18T23:08:42.058 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:42.071 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-18T23:08:42.157 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-18T23:08:42.192 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-18T23:08:42.287 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-04-18T23:08:42.399 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:42.437 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-18T23:08:42.451 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T23:08:42.486 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-18T23:08:42.508 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-18T23:08:42.550 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T23:08:42.578 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-18T23:08:42.676 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-18T23:08:42.704 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T23:08:42.734 INFO:teuthology.orchestra.run.smithi120.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-04-18T23:08:42.768 INFO:teuthology.orchestra.run.smithi120.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-04-18T23:08:42.818 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-18T23:08:42.824 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-04-18T23:08:42.925 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-04-18T23:08:42.997 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-18T23:08:43.036 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:43.078 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-18T23:08:43.145 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-18T23:08:43.239 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-18T23:08:43.276 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:43.337 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-18T23:08:43.387 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T23:08:43.411 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-04-18T23:08:43.482 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:43.605 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-18T23:08:43.608 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-18T23:08:43.647 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-18T23:08:43.687 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-04-18T23:08:43.856 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-18T23:08:43.909 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-18T23:08:43.947 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-04-18T23:08:43.956 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up zip (3.0-12build2) ... 2024-04-18T23:08:44.082 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T23:08:44.106 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T23:08:44.160 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-18T23:08:44.224 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-18T23:08:44.239 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up zip (3.0-12build2) ... 2024-04-18T23:08:44.365 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-04-18T23:08:44.411 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-18T23:08:44.517 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T23:08:44.651 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T23:08:44.662 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-18T23:08:44.759 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T23:08:44.914 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-18T23:08:44.944 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-18T23:08:45.018 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T23:08:45.051 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-04-18T23:08:45.120 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-18T23:08:45.157 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T23:08:45.220 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T23:08:45.262 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-18T23:08:45.274 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-18T23:08:45.327 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-04-18T23:08:45.453 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-04-18T23:08:45.505 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-18T23:08:45.521 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T23:08:45.579 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-04-18T23:08:45.585 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T23:08:45.664 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-18T23:08:45.710 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T23:08:45.790 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:45.879 INFO:teuthology.orchestra.run.smithi136.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-18T23:08:45.905 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-04-18T23:08:45.914 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-18T23:08:45.986 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-18T23:08:46.116 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T23:08:46.165 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T23:08:46.217 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T23:08:46.278 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-18T23:08:46.290 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-04-18T23:08:46.529 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T23:08:46.533 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-04-18T23:08:46.533 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T23:08:46.602 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-18T23:08:46.647 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-18T23:08:46.765 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:46.818 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T23:08:46.839 INFO:teuthology.orchestra.run.smithi073.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-18T23:08:46.869 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-18T23:08:46.885 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-04-18T23:08:46.996 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-18T23:08:47.108 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T23:08:47.136 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-18T23:08:47.234 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T23:08:47.255 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:47.295 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-04-18T23:08:47.362 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-18T23:08:47.364 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T23:08:47.488 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:47.619 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-18T23:08:47.656 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T23:08:47.789 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-idna (3.3-1) ... 2024-04-18T23:08:47.813 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-04-18T23:08:47.913 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-18T23:08:47.933 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T23:08:48.038 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-18T23:08:48.064 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-04-18T23:08:48.124 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T23:08:48.184 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-18T23:08:48.190 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:48.305 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:48.318 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-18T23:08:48.423 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T23:08:48.466 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-idna (3.3-1) ... 2024-04-18T23:08:48.594 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-18T23:08:48.733 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T23:08:48.776 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-04-18T23:08:48.853 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T23:08:49.034 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T23:08:49.304 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T23:08:49.318 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-18T23:08:49.423 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-18T23:08:49.461 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-18T23:08:49.539 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:49.673 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-18T23:08:49.711 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-18T23:08:49.825 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-18T23:08:49.891 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-04-18T23:08:49.958 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T23:08:50.084 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-18T23:08:50.093 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T23:08:50.138 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T23:08:50.150 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:50.210 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T23:08:50.285 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-toml (0.10.2-1) ... 2024-04-18T23:08:50.429 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T23:08:50.491 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-18T23:08:50.511 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-18T23:08:50.545 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-04-18T23:08:50.546 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:50.655 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:50.671 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-04-18T23:08:50.755 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-18T23:08:50.765 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-18T23:08:50.797 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-04-18T23:08:50.868 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-18T23:08:50.997 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:51.090 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-04-18T23:08:51.109 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:51.122 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-18T23:08:51.144 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T23:08:51.235 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:51.341 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-04-18T23:08:51.361 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-18T23:08:51.416 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-18T23:08:51.463 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T23:08:51.589 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:51.593 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:51.629 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:51.683 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-18T23:08:51.724 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:51.727 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-04-18T23:08:51.755 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:51.850 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-18T23:08:51.881 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:51.951 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-18T23:08:52.020 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-04-18T23:08:52.084 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T23:08:52.084 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T23:08:52.161 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:52.202 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-18T23:08:52.262 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:52.271 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-04-18T23:08:52.388 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-18T23:08:52.428 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-18T23:08:52.531 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-04-18T23:08:52.540 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:52.650 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:52.664 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:52.679 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T23:08:52.768 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:52.782 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-04-18T23:08:52.799 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:52.805 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-18T23:08:52.894 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T23:08:52.925 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:53.025 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-04-18T23:08:53.081 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T23:08:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T23:08:53.142 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T23:08:53.207 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T23:08:53.276 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-04-18T23:08:53.411 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-04-18T23:08:53.492 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-18T23:08:53.643 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:53.670 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-04-18T23:08:53.761 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:53.796 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-04-18T23:08:53.801 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-18T23:08:53.862 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:53.979 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T23:08:54.052 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T23:08:54.072 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:54.106 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-04-18T23:08:54.153 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-18T23:08:54.181 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:54.254 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:54.307 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:54.344 INFO:teuthology.orchestra.run.smithi142.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-18T23:08:54.415 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-04-18T23:08:54.433 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:54.567 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:54.581 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T23:08:54.666 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-04-18T23:08:54.732 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T23:08:54.776 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-04-18T23:08:54.910 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-04-18T23:08:55.000 INFO:teuthology.orchestra.run.smithi120.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-04-18T23:08:55.093 INFO:teuthology.orchestra.run.smithi136.stdout:Adding group ceph....done 2024-04-18T23:08:55.125 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-18T23:08:55.133 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:55.229 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-04-18T23:08:55.259 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:55.347 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-04-18T23:08:55.384 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-18T23:08:55.385 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:55.445 INFO:teuthology.orchestra.run.smithi136.stdout:Adding system user ceph....done 2024-04-18T23:08:55.504 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-18T23:08:55.511 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:55.555 INFO:teuthology.orchestra.run.smithi136.stdout:Setting system user ceph properties....done 2024-04-18T23:08:55.572 INFO:teuthology.orchestra.run.smithi136.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-18T23:08:55.637 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:55.707 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-18T23:08:55.732 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-04-18T23:08:55.763 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:55.889 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T23:08:55.983 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up lua-any (27ubuntu1) ... 2024-04-18T23:08:56.060 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-18T23:08:56.109 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-portend (3.0.0-1) ... 2024-04-18T23:08:56.139 INFO:teuthology.orchestra.run.smithi073.stdout:Adding group ceph....done 2024-04-18T23:08:56.199 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T23:08:56.369 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-04-18T23:08:56.470 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-04-18T23:08:56.481 INFO:teuthology.orchestra.run.smithi073.stdout:Adding system user ceph....done 2024-04-18T23:08:56.500 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T23:08:56.518 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:56.608 INFO:teuthology.orchestra.run.smithi073.stdout:Setting system user ceph properties....done 2024-04-18T23:08:56.628 INFO:teuthology.orchestra.run.smithi073.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-18T23:08:56.644 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:56.766 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-18T23:08:56.771 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-04-18T23:08:56.776 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-18T23:08:56.885 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-18T23:08:57.064 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-04-18T23:08:57.093 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T23:08:57.093 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T23:08:57.093 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-18T23:08:57.136 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-18T23:08:57.373 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-04-18T23:08:57.396 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T23:08:57.474 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-rsa (4.8-1) ... 2024-04-18T23:08:57.571 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:57.680 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:57.725 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-04-18T23:08:57.782 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-18T23:08:57.863 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:58.002 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-04-18T23:08:58.144 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T23:08:58.145 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T23:08:58.167 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-18T23:08:58.201 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-18T23:08:58.339 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-04-18T23:08:58.452 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T23:08:58.673 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:58.741 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-04-18T23:08:58.771 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T23:08:58.818 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:58.895 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T23:08:58.895 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T23:08:58.897 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:59.008 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-04-18T23:08:59.023 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:59.149 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-18T23:08:59.342 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:59.353 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-04-18T23:08:59.415 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-18T23:08:59.469 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:59.479 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up librados2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:59.563 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T23:08:59.563 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T23:08:59.570 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:59.606 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libsqlite3-mod-ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:59.688 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-18T23:08:59.723 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-04-18T23:08:59.896 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:08:59.973 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.052 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libcephfs2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.099 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.113 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.130 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T23:09:00.130 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T23:09:00.178 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libradosstriper1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.225 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.304 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-04-18T23:09:00.386 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T23:09:00.386 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T23:09:00.435 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T23:09:00.435 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T23:09:00.581 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up librbd1 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.652 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.707 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-modules-core (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.841 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.848 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.854 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T23:09:00.854 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T23:09:00.917 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:00.957 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:01.027 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:01.055 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T23:09:01.055 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-04-18T23:09:01.109 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:01.128 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:01.246 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T23:09:01.319 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T23:09:01.320 INFO:teuthology.orchestra.run.smithi136.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T23:09:01.328 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:01.559 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up libcephfs-dev (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:01.605 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T23:09:01.605 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T23:09:01.685 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-rados (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:01.752 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:01.787 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up librgw2 (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:01.888 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-04-18T23:09:01.894 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:02.020 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:02.104 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:02.171 INFO:teuthology.orchestra.run.smithi136.stdout:Setting up ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:02.230 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:02.373 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:02.457 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:02.498 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-18T23:09:02.583 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T23:09:02.589 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T23:09:02.589 INFO:teuthology.orchestra.run.smithi073.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T23:09:02.692 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:02.719 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T23:09:03.076 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.082 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.133 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-rbd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.177 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.234 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.251 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up rbd-fuse (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.278 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.343 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.377 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-rgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.478 INFO:teuthology.orchestra.run.smithi073.stdout:Setting up ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.503 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up python3-cephfs (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.630 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-common (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:03.761 INFO:teuthology.orchestra.run.smithi136.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T23:09:03.787 INFO:teuthology.orchestra.run.smithi142.stdout:Adding group ceph....done 2024-04-18T23:09:03.864 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-18T23:09:03.973 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T23:09:04.100 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T23:09:04.122 INFO:teuthology.orchestra.run.smithi120.stdout:Adding group ceph....done 2024-04-18T23:09:04.139 INFO:teuthology.orchestra.run.smithi142.stdout:Adding system user ceph....done 2024-04-18T23:09:04.283 INFO:teuthology.orchestra.run.smithi142.stdout:Setting system user ceph properties....done 2024-04-18T23:09:04.295 INFO:teuthology.orchestra.run.smithi142.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-18T23:09:04.403 INFO:teuthology.orchestra.run.smithi136.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T23:09:04.410 DEBUG:teuthology.parallel:result is None 2024-04-18T23:09:04.430 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-18T23:09:04.508 INFO:teuthology.orchestra.run.smithi120.stdout:Adding system user ceph....done 2024-04-18T23:09:04.669 INFO:teuthology.orchestra.run.smithi120.stdout:Setting system user ceph properties....done 2024-04-18T23:09:04.689 INFO:teuthology.orchestra.run.smithi120.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-04-18T23:09:04.736 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-18T23:09:04.820 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-04-18T23:09:05.117 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-04-18T23:09:05.192 INFO:teuthology.orchestra.run.smithi073.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T23:09:05.221 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:05.347 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:05.557 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-test (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:05.691 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up radosgw (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:05.808 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T23:09:05.808 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T23:09:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T23:09:05.828 DEBUG:teuthology.parallel:result is None 2024-04-18T23:09:06.126 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T23:09:06.126 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-04-18T23:09:06.274 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:06.551 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-base (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:06.623 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-18T23:09:06.900 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-04-18T23:09:07.109 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:07.327 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T23:09:07.327 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T23:09:07.388 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mds (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:07.618 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T23:09:07.618 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-04-18T23:09:07.819 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:08.020 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T23:09:08.020 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T23:09:08.089 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:08.309 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T23:09:08.314 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-04-18T23:09:08.495 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:08.766 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T23:09:08.766 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T23:09:08.790 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-osd (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:09.041 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T23:09:09.041 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-04-18T23:09:09.247 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:09.373 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:09.524 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:09.525 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-k8sevents (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:09.651 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:09.738 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T23:09:09.738 INFO:teuthology.orchestra.run.smithi142.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T23:09:09.803 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mon (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:10.018 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T23:09:10.019 INFO:teuthology.orchestra.run.smithi120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-04-18T23:09:10.217 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:10.359 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:10.485 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:10.537 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-cephadm (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:10.645 INFO:teuthology.orchestra.run.smithi142.stdout:Setting up ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:10.688 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:10.814 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-mgr-dashboard (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:10.957 INFO:teuthology.orchestra.run.smithi120.stdout:Setting up ceph-volume (19.0.0-2369-g1a6a3e74-1jammy) ... 2024-04-18T23:09:11.005 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-18T23:09:11.107 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T23:09:11.251 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T23:09:11.285 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-04-18T23:09:11.403 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-04-18T23:09:11.555 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-04-18T23:09:12.285 INFO:teuthology.orchestra.run.smithi142.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T23:09:12.607 INFO:teuthology.orchestra.run.smithi120.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-04-18T23:09:12.917 INFO:teuthology.orchestra.run.smithi142.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T23:09:12.922 DEBUG:teuthology.parallel:result is None 2024-04-18T23:09:13.219 INFO:teuthology.orchestra.run.smithi120.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-04-18T23:09:13.223 DEBUG:teuthology.parallel:result is None 2024-04-18T23:09:13.224 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T23:09:13.446 DEBUG:teuthology.orchestra.run.smithi073:> dpkg-query -W -f '${Version}' ceph 2024-04-18T23:09:13.465 INFO:teuthology.orchestra.run.smithi073.stdout:19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:09:13.466 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:09:13.466 INFO:teuthology.task.install:The correct ceph version 19.0.0-2369-g1a6a3e74-1jammy is installed. 2024-04-18T23:09:13.468 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T23:09:13.718 DEBUG:teuthology.orchestra.run.smithi120:> dpkg-query -W -f '${Version}' ceph 2024-04-18T23:09:13.737 INFO:teuthology.orchestra.run.smithi120.stdout:19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:09:13.738 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:09:13.738 INFO:teuthology.task.install:The correct ceph version 19.0.0-2369-g1a6a3e74-1jammy is installed. 2024-04-18T23:09:13.740 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T23:09:13.879 DEBUG:teuthology.orchestra.run.smithi136:> dpkg-query -W -f '${Version}' ceph 2024-04-18T23:09:13.898 INFO:teuthology.orchestra.run.smithi136.stdout:19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:09:13.898 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:09:13.898 INFO:teuthology.task.install:The correct ceph version 19.0.0-2369-g1a6a3e74-1jammy is installed. 2024-04-18T23:09:13.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=1a6a3e742fd81c6d70be70552499bff767652c32 2024-04-18T23:09:14.136 DEBUG:teuthology.orchestra.run.smithi142:> dpkg-query -W -f '${Version}' ceph 2024-04-18T23:09:14.155 INFO:teuthology.orchestra.run.smithi142.stdout:19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:09:14.155 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2369-g1a6a3e74-1jammy 2024-04-18T23:09:14.155 INFO:teuthology.task.install:The correct ceph version 19.0.0-2369-g1a6a3e74-1jammy is installed. 2024-04-18T23:09:14.157 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-18T23:09:14.157 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:14.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-18T23:09:14.173 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:09:14.173 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-18T23:09:14.187 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:09:14.187 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-18T23:09:14.202 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:09:14.202 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-18T23:09:14.213 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-18T23:09:14.214 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:14.214 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2024-04-18T23:09:14.228 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-18T23:09:14.282 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:09:14.282 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/daemon-helper 2024-04-18T23:09:14.295 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-18T23:09:14.349 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:09:14.349 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/daemon-helper 2024-04-18T23:09:14.363 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-18T23:09:14.418 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:09:14.418 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/daemon-helper 2024-04-18T23:09:14.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-18T23:09:14.484 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-18T23:09:14.485 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:14.485 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-18T23:09:14.498 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-18T23:09:14.551 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:09:14.551 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-18T23:09:14.565 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-18T23:09:14.617 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:09:14.617 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-18T23:09:14.631 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-18T23:09:14.686 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:09:14.686 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-18T23:09:14.699 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-18T23:09:14.751 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-18T23:09:14.752 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:14.752 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2024-04-18T23:09:14.765 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-18T23:09:14.820 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:09:14.820 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/usr/bin/stdin-killer 2024-04-18T23:09:14.834 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-18T23:09:14.890 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:09:14.890 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/usr/bin/stdin-killer 2024-04-18T23:09:14.903 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-18T23:09:14.959 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:09:14.959 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/usr/bin/stdin-killer 2024-04-18T23:09:14.972 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-18T23:09:15.028 INFO:teuthology.run_tasks:Running task ceph... 2024-04-18T23:09:15.109 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-18T23:09:15.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /var/log/ceph 2024-04-18T23:09:15.112 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 777 /var/log/ceph 2024-04-18T23:09:15.114 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 777 /var/log/ceph 2024-04-18T23:09:15.117 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 777 /var/log/ceph 2024-04-18T23:09:15.129 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-18T23:09:15.129 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-18T23:09:15.169 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-18T23:09:15.171 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-18T23:09:15.173 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-18T23:09:15.185 INFO:tasks.ceph:Creating extra log directories... 2024-04-18T23:09:15.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-18T23:09:15.221 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-18T23:09:15.227 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-18T23:09:15.229 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-18T23:09:15.255 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-18T23:09:15.255 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-18T23:09:15.255 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678', 'branch': 'squid', 'description': 'rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7662678', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd0bf63dd179c05751e4be49477389aa290c73089'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 101, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'mgr.y', 'osd.0', 'osd.4', 'osd.8', 'osd.12'], ['mon.b', 'osd.1', 'osd.5', 'osd.9', 'osd.13'], ['mon.c', 'osd.2', 'osd.6', 'osd.10', 'osd.14'], ['mgr.x', 'osd.3', 'osd.7', 'osd.11', 'osd.15', 'client.0']], 'seed': 6512, 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32', 'sleep_before_teardown': 0, 'subset': '28391/100000', 'suite': 'rados', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_d0bf63dd179c05751e4be49477389aa290c73089/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'd0bf63dd179c05751e4be49477389aa290c73089', 'targets': {'smithi073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGfqB4DVrnrJLzirGB5de2wcaY2pBc8V5fbOxVQlrXlBFgJt0i2bqXUIiqLLPK8FpjoFBa9vi36ZgZIkt+ZHVv8=', 'smithi120.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPRCs7ZAWG0V5CWjphJNZmIQrAkvK0fg2glJpBpD5Fp2CFpqQ6vpDVnzHDf4qGqDJTNqf+ie4TgmPzOEYLxr954=', 'smithi136.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBv0kAkKsnr9mYTUL9yt5H8GNLJWUrq7NLlyI7L0+mFDC3b37uB3afLviw4z5pIgEMde9jD1QkY56bSIMSStOrI=', 'smithi142.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLWvCQrq9gAqochkHl1mWyigv/lu3X+lFu5BOi/j1f5KDFqtCcpQpXO377WTxoCRP1FN3w8Cn6C35ORnDJvP1QU='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'global': {'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32', 'cluster': 'ceph'}}, {'thrashosds': {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200}}, {'rados': {'clients': ['client.0'], 'ec_pool': True, 'erasure_code_profile': {'c': 2, 'crush-failure-domain': 'osd', 'k': 4, 'm': 3, 'name': 'shecprofile', 'plugin': 'shec'}, 'objects': 50, 'op_weights': {'append': 100, 'copy_from': 50, 'delete': 50, 'read': 100, 'rmattr': 25, 'rollback': 50, 'setattr': 25, 'snap_create': 50, 'snap_remove': 50, 'write': 0}, 'ops': 400, 'write_append_excl': False}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-18_21:00:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-18T23:09:15.256 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-18T23:09:15.277 DEBUG:teuthology.orchestra.run.smithi120:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-18T23:09:15.283 DEBUG:teuthology.orchestra.run.smithi136:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-18T23:09:15.285 DEBUG:teuthology.orchestra.run.smithi142:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-18T23:09:15.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/run/ceph 2024-04-18T23:09:15.325 DEBUG:teuthology.orchestra.run.smithi120:> sudo install -d -m0777 -- /var/run/ceph 2024-04-18T23:09:15.331 DEBUG:teuthology.orchestra.run.smithi136:> sudo install -d -m0777 -- /var/run/ceph 2024-04-18T23:09:15.333 DEBUG:teuthology.orchestra.run.smithi142:> sudo install -d -m0777 -- /var/run/ceph 2024-04-18T23:09:15.346 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:15.346 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2024-04-18T23:09:15.384 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-18T23:09:15.384 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2024-04-18T23:09:15.432 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-18T23:09:15.432 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:15.432 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 876 Links: 1 2024-04-18T23:09:15.432 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:15.432 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-18 23:03:45.093741916 +0000 2024-04-18T23:09:15.432 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-18 23:03:44.829744275 +0000 2024-04-18T23:09:15.432 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-18 23:03:44.829744275 +0000 2024-04-18T23:09:15.432 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-04-18T23:09:15.433 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-18T23:09:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-18T23:09:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-18T23:09:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000381456 s, 1.3 MB/s 2024-04-18T23:09:15.487 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-18T23:09:15.537 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2024-04-18T23:09:15.584 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-18T23:09:15.584 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:15.584 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-04-18T23:09:15.584 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:15.584 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-18 23:03:45.301740058 +0000 2024-04-18T23:09:15.584 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-18 23:03:45.297740094 +0000 2024-04-18T23:09:15.585 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-18 23:03:45.297740094 +0000 2024-04-18T23:09:15.585 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-04-18T23:09:15.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-18T23:09:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-18T23:09:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-18T23:09:15.638 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.00036968 s, 1.4 MB/s 2024-04-18T23:09:15.639 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-18T23:09:15.689 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2024-04-18T23:09:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-18T23:09:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:15.739 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 913 Links: 1 2024-04-18T23:09:15.739 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:15.739 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-18 23:03:46.049733375 +0000 2024-04-18T23:09:15.739 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-18 23:03:45.785735734 +0000 2024-04-18T23:09:15.739 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-18 23:03:45.785735734 +0000 2024-04-18T23:09:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-04-18T23:09:15.739 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-18T23:09:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-18T23:09:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-18T23:09:15.790 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000380342 s, 1.3 MB/s 2024-04-18T23:09:15.791 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-18T23:09:15.841 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2024-04-18T23:09:15.888 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-18T23:09:15.888 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:15.888 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 5h/5d Inode: 928 Links: 1 2024-04-18T23:09:15.888 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:15.888 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2024-04-18 23:03:46.301731125 +0000 2024-04-18T23:09:15.888 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2024-04-18 23:03:46.297731159 +0000 2024-04-18T23:09:15.888 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2024-04-18 23:03:46.297731159 +0000 2024-04-18T23:09:15.888 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2024-04-18T23:09:15.888 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-18T23:09:15.943 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2024-04-18T23:09:15.943 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2024-04-18T23:09:15.943 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000368669 s, 1.4 MB/s 2024-04-18T23:09:15.944 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-18T23:09:15.993 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:15.993 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:09:15.993 DEBUG:teuthology.orchestra.run.smithi120:> dd if=/scratch_devs of=/dev/stdout 2024-04-18T23:09:15.999 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-18T23:09:15.999 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_1 2024-04-18T23:09:16.046 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-18T23:09:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:16.047 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 879 Links: 1 2024-04-18T23:09:16.047 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:16.047 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-18 23:03:45.433322810 +0000 2024-04-18T23:09:16.047 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-18 23:03:45.165325194 +0000 2024-04-18T23:09:16.047 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-18 23:03:45.165325194 +0000 2024-04-18T23:09:16.047 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2024-04-18T23:09:16.047 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-18T23:09:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-18T23:09:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-18T23:09:16.101 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000427338 s, 1.2 MB/s 2024-04-18T23:09:16.102 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-18T23:09:16.151 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_2 2024-04-18T23:09:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-18T23:09:16.197 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:16.197 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-04-18T23:09:16.197 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:16.198 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-18 23:03:45.913318537 +0000 2024-04-18T23:09:16.198 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-18 23:03:45.649320887 +0000 2024-04-18T23:09:16.198 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-18 23:03:45.649320887 +0000 2024-04-18T23:09:16.198 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2024-04-18T23:09:16.198 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-18T23:09:16.252 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-18T23:09:16.252 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-18T23:09:16.252 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000405454 s, 1.3 MB/s 2024-04-18T23:09:16.253 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-18T23:09:16.303 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_3 2024-04-18T23:09:16.350 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-18T23:09:16.350 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:16.350 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 908 Links: 1 2024-04-18T23:09:16.350 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:16.350 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-18 23:03:46.433313910 +0000 2024-04-18T23:09:16.350 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-18 23:03:46.165316295 +0000 2024-04-18T23:09:16.350 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-18 23:03:46.165316295 +0000 2024-04-18T23:09:16.350 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2024-04-18T23:09:16.350 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-18T23:09:16.403 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-18T23:09:16.404 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-18T23:09:16.404 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000407014 s, 1.3 MB/s 2024-04-18T23:09:16.405 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-18T23:09:16.452 DEBUG:teuthology.orchestra.run.smithi120:> stat /dev/vg_nvme/lv_4 2024-04-18T23:09:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-18T23:09:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:16.498 INFO:teuthology.orchestra.run.smithi120.stdout:Device: 5h/5d Inode: 929 Links: 1 2024-04-18T23:09:16.498 INFO:teuthology.orchestra.run.smithi120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:16.498 INFO:teuthology.orchestra.run.smithi120.stdout:Access: 2024-04-18 23:03:46.677311738 +0000 2024-04-18T23:09:16.498 INFO:teuthology.orchestra.run.smithi120.stdout:Modify: 2024-04-18 23:03:46.673311774 +0000 2024-04-18T23:09:16.498 INFO:teuthology.orchestra.run.smithi120.stdout:Change: 2024-04-18 23:03:46.673311774 +0000 2024-04-18T23:09:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: Birth: - 2024-04-18T23:09:16.498 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-18T23:09:16.552 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records in 2024-04-18T23:09:16.552 INFO:teuthology.orchestra.run.smithi120.stderr:1+0 records out 2024-04-18T23:09:16.552 INFO:teuthology.orchestra.run.smithi120.stderr:512 bytes copied, 0.000380531 s, 1.3 MB/s 2024-04-18T23:09:16.554 DEBUG:teuthology.orchestra.run.smithi120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-18T23:09:16.603 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:16.603 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:09:16.603 DEBUG:teuthology.orchestra.run.smithi136:> dd if=/scratch_devs of=/dev/stdout 2024-04-18T23:09:16.609 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-18T23:09:16.609 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_1 2024-04-18T23:09:16.655 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-18T23:09:16.655 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:16.655 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 882 Links: 1 2024-04-18T23:09:16.655 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:16.655 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-18 23:03:46.422715360 +0000 2024-04-18T23:09:16.655 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-18 23:03:46.178717539 +0000 2024-04-18T23:09:16.655 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-18 23:03:46.178717539 +0000 2024-04-18T23:09:16.656 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-04-18T23:09:16.656 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-18T23:09:16.709 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-18T23:09:16.709 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-18T23:09:16.710 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000443911 s, 1.2 MB/s 2024-04-18T23:09:16.711 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-18T23:09:16.760 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_2 2024-04-18T23:09:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-18T23:09:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:16.807 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 896 Links: 1 2024-04-18T23:09:16.807 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:16.807 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-18 23:03:46.934710786 +0000 2024-04-18T23:09:16.807 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-18 23:03:46.662713216 +0000 2024-04-18T23:09:16.807 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-18 23:03:46.662713216 +0000 2024-04-18T23:09:16.807 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-04-18T23:09:16.807 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-18T23:09:16.861 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-18T23:09:16.861 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-18T23:09:16.861 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000405427 s, 1.3 MB/s 2024-04-18T23:09:16.862 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-18T23:09:16.912 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_3 2024-04-18T23:09:16.958 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-18T23:09:16.958 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:16.958 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 917 Links: 1 2024-04-18T23:09:16.959 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:16.959 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-18 23:03:47.426706391 +0000 2024-04-18T23:09:16.959 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-18 23:03:47.178708606 +0000 2024-04-18T23:09:16.959 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-18 23:03:47.178708606 +0000 2024-04-18T23:09:16.959 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-04-18T23:09:16.959 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-18T23:09:17.013 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-18T23:09:17.013 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-18T23:09:17.013 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000432217 s, 1.2 MB/s 2024-04-18T23:09:17.014 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-18T23:09:17.064 DEBUG:teuthology.orchestra.run.smithi136:> stat /dev/vg_nvme/lv_4 2024-04-18T23:09:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-18T23:09:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:17.111 INFO:teuthology.orchestra.run.smithi136.stdout:Device: 5h/5d Inode: 932 Links: 1 2024-04-18T23:09:17.111 INFO:teuthology.orchestra.run.smithi136.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:17.111 INFO:teuthology.orchestra.run.smithi136.stdout:Access: 2024-04-18 23:03:47.926701926 +0000 2024-04-18T23:09:17.111 INFO:teuthology.orchestra.run.smithi136.stdout:Modify: 2024-04-18 23:03:47.662704284 +0000 2024-04-18T23:09:17.111 INFO:teuthology.orchestra.run.smithi136.stdout:Change: 2024-04-18 23:03:47.662704284 +0000 2024-04-18T23:09:17.111 INFO:teuthology.orchestra.run.smithi136.stdout: Birth: - 2024-04-18T23:09:17.111 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-18T23:09:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records in 2024-04-18T23:09:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:1+0 records out 2024-04-18T23:09:17.164 INFO:teuthology.orchestra.run.smithi136.stderr:512 bytes copied, 0.000344295 s, 1.5 MB/s 2024-04-18T23:09:17.166 DEBUG:teuthology.orchestra.run.smithi136:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-18T23:09:17.215 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:17.216 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:09:17.216 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/scratch_devs of=/dev/stdout 2024-04-18T23:09:17.221 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-18T23:09:17.221 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_1 2024-04-18T23:09:17.270 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-18T23:09:17.270 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:17.270 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 882 Links: 1 2024-04-18T23:09:17.270 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:17.270 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-04-18 23:03:47.569190601 +0000 2024-04-18T23:09:17.270 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-04-18 23:03:47.321192816 +0000 2024-04-18T23:09:17.270 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-04-18 23:03:47.321192816 +0000 2024-04-18T23:09:17.270 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-04-18T23:09:17.270 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-18T23:09:17.324 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-04-18T23:09:17.324 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-04-18T23:09:17.324 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000455543 s, 1.1 MB/s 2024-04-18T23:09:17.325 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-18T23:09:17.374 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_2 2024-04-18T23:09:17.421 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-18T23:09:17.421 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:17.421 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 895 Links: 1 2024-04-18T23:09:17.421 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:17.421 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-04-18 23:03:47.789188636 +0000 2024-04-18T23:09:17.421 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-04-18 23:03:47.785188672 +0000 2024-04-18T23:09:17.422 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-04-18 23:03:47.785188672 +0000 2024-04-18T23:09:17.422 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-04-18T23:09:17.422 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-18T23:09:17.475 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-04-18T23:09:17.475 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-04-18T23:09:17.476 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00042 s, 1.2 MB/s 2024-04-18T23:09:17.477 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-18T23:09:17.526 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_3 2024-04-18T23:09:17.574 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-18T23:09:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:17.578 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 910 Links: 1 2024-04-18T23:09:17.578 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:17.578 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-04-18 23:03:48.273184312 +0000 2024-04-18T23:09:17.578 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-04-18 23:03:48.269184349 +0000 2024-04-18T23:09:17.578 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-04-18 23:03:48.269184349 +0000 2024-04-18T23:09:17.578 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-04-18T23:09:17.579 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-18T23:09:17.628 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-04-18T23:09:17.628 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-04-18T23:09:17.628 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.00040629 s, 1.3 MB/s 2024-04-18T23:09:17.629 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-18T23:09:17.678 DEBUG:teuthology.orchestra.run.smithi142:> stat /dev/vg_nvme/lv_4 2024-04-18T23:09:17.725 INFO:teuthology.orchestra.run.smithi142.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-18T23:09:17.725 INFO:teuthology.orchestra.run.smithi142.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-18T23:09:17.725 INFO:teuthology.orchestra.run.smithi142.stdout:Device: 5h/5d Inode: 928 Links: 1 2024-04-18T23:09:17.725 INFO:teuthology.orchestra.run.smithi142.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-18T23:09:17.725 INFO:teuthology.orchestra.run.smithi142.stdout:Access: 2024-04-18 23:03:49.089177025 +0000 2024-04-18T23:09:17.725 INFO:teuthology.orchestra.run.smithi142.stdout:Modify: 2024-04-18 23:03:48.797179632 +0000 2024-04-18T23:09:17.725 INFO:teuthology.orchestra.run.smithi142.stdout:Change: 2024-04-18 23:03:48.797179632 +0000 2024-04-18T23:09:17.725 INFO:teuthology.orchestra.run.smithi142.stdout: Birth: - 2024-04-18T23:09:17.725 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-18T23:09:17.779 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records in 2024-04-18T23:09:17.780 INFO:teuthology.orchestra.run.smithi142.stderr:1+0 records out 2024-04-18T23:09:17.780 INFO:teuthology.orchestra.run.smithi142.stderr:512 bytes copied, 0.000375791 s, 1.4 MB/s 2024-04-18T23:09:17.781 DEBUG:teuthology.orchestra.run.smithi142:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-18T23:09:17.830 INFO:tasks.ceph:osd dev map: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:17.830 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-04-18T23:09:17.830 INFO:tasks.ceph:Generating config... 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] debug monc = 20 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-18T23:09:17.832 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd debug reject backfill probability = 0.3 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd max backfills = 3 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd scrub max interval = 120 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd scrub min interval = 60 2024-04-18T23:09:17.833 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-18T23:09:17.833 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-04-18T23:09:17.833 INFO:tasks.ceph:[global] osd debug inject dispatch delay duration = 0.1 2024-04-18T23:09:17.834 INFO:tasks.ceph:[global] osd debug inject dispatch delay probability = 0.1 2024-04-18T23:09:17.834 INFO:tasks.ceph:[global] osd_async_recovery_min_cost = 1 2024-04-18T23:09:17.834 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-18T23:09:17.834 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-18T23:09:17.834 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-18T23:09:17.834 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-18T23:09:17.834 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-18T23:09:17.834 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-04-18T23:09:17.834 INFO:tasks.ceph:Setting up mon.a... 2024-04-18T23:09:17.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-18T23:09:17.868 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.keyring 2024-04-18T23:09:17.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-18T23:09:17.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T23:09:18.011 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.73'), ('mon.b', '172.21.15.120'), ('mon.c', '172.21.15.136')] 2024-04-18T23:09:18.011 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.73,172.21.15.120,172.21.15.136', 'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': 120, 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon scrub interval': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}} 2024-04-18T23:09:18.012 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:18.012 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-18T23:09:18.060 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.73 --add b 172.21.15.120 --add c 172.21.15.136 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:18.141 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:18.141 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: generated fsid b8d0ffd9-6668-442b-a039-a24cea734f3e 2024-04-18T23:09:18.141 INFO:teuthology.orchestra.run.smithi073.stdout:setting min_mon_release = quincy 2024-04-18T23:09:18.141 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 0 2024-04-18T23:09:18.142 INFO:teuthology.orchestra.run.smithi073.stdout:fsid b8d0ffd9-6668-442b-a039-a24cea734f3e 2024-04-18T23:09:18.142 INFO:teuthology.orchestra.run.smithi073.stdout:last_changed 2024-04-18T23:09:18.142760+0000 2024-04-18T23:09:18.142 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-04-18T23:09:18.142760+0000 2024-04-18T23:09:18.142 INFO:teuthology.orchestra.run.smithi073.stdout:min_mon_release 17 (quincy) 2024-04-18T23:09:18.142 INFO:teuthology.orchestra.run.smithi073.stdout:election_strategy: 3 2024-04-18T23:09:18.142 INFO:teuthology.orchestra.run.smithi073.stdout:0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2024-04-18T23:09:18.142 INFO:teuthology.orchestra.run.smithi073.stdout:1: [v2:172.21.15.120:3300/0,v1:172.21.15.120:6789/0] mon.b 2024-04-18T23:09:18.142 INFO:teuthology.orchestra.run.smithi073.stdout:2: [v2:172.21.15.136:3300/0,v1:172.21.15.136:6789/0] mon.c 2024-04-18T23:09:18.142 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-18T23:09:18.143 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-18T23:09:18.192 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID b8d0ffd9-6668-442b-a039-a24cea734f3e... 2024-04-18T23:09:18.193 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-18T23:09:18.237 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-18T23:09:18.239 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-18T23:09:18.241 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-18T23:09:18.265 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2024-04-18T23:09:18.265 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2024-04-18T23:09:18.265 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-18T23:09:18.265 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2024-04-18T23:09:18.266 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2024-04-18T23:09:18.267 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.73,172.21.15.120,172.21.15.136" 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: mon election default strategy = 3 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug inject dispatch delay duration = 0.1 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug inject dispatch delay probability = 0.1 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: osd_async_recovery_min_cost = 1 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = b8d0ffd9-6668-442b-a039-a24cea734f3e 2024-04-18T23:09:18.268 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 120 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock profile = high_recovery_ops 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug trim objects = true 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.269 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: debug monc = 20 2024-04-18T23:09:18.270 INFO:teuthology.orchestra.run.smithi073.stdout: bdev async discard = True 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: bdev enable discard = True 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore allocator = bitmap 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore zero block detection = True 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 20 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 20 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 10 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: osd blocked scrub grace period = 3600 2024-04-18T23:09:18.271 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug reject backfill probability = 0.3 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify cached snaps = True 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify missing on start = True 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd max backfills = 3 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd max markdown count = 1000 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd mclock override recovery settings = True 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd op queue = debug_random 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd op queue cut off = debug_random 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub min interval = 60 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: osd shutdown pgref assert = True 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2024-04-18T23:09:18.272 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2024-04-18T23:09:18.273 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: # 1m isn't quite enough 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: mon_down_mkfs_grace = 2m 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_filestore_osds = false 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: mon scrub interval = 300 2024-04-18T23:09:18.274 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:09:18.275 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2024-04-18T23:09:18.275 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2024-04-18T23:09:18.275 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2024-04-18T23:09:18.275 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2024-04-18T23:09:18.275 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-18T23:09:18.275 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-18T23:09:18.275 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2024-04-18T23:09:18.275 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2024-04-18T23:09:18.275 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2024-04-18T23:09:18.276 INFO:teuthology.orchestra.run.smithi142.stdout:[global] 2024-04-18T23:09:18.276 INFO:teuthology.orchestra.run.smithi142.stdout: chdir = "" 2024-04-18T23:09:18.276 INFO:teuthology.orchestra.run.smithi142.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-18T23:09:18.276 INFO:teuthology.orchestra.run.smithi142.stdout: auth supported = cephx 2024-04-18T23:09:18.276 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.276 INFO:teuthology.orchestra.run.smithi142.stdout: filestore xattr use omap = true 2024-04-18T23:09:18.276 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.276 INFO:teuthology.orchestra.run.smithi142.stdout: mon clock drift allowed = 1.000 2024-04-18T23:09:18.276 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: osd crush chooseleaf type = 0 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: auth debug = true 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on old message = true 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: ms die on bug = true 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: mon pg warn max object skew = 0 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: # disable pg_autoscaler by default for new pools 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default size = 2 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow primary affinity = true 2024-04-18T23:09:18.277 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd allow pg remap = true 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on legacy crush tunables = false 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on crush straw calc version zero = false 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on no sortbitwise = false 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on osd down out interval zero = false 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: mon warn on too few osds = false 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: mon_allow_pool_size_one = true 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: osd default data pool replay window = 5 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: mon allow pool delete = true 2024-04-18T23:09:18.278 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: mon cluster log file level = debug 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: debug asserts on shutdown = true 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: mon health detail to clog = false 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: mon host = "172.21.15.73,172.21.15.120,172.21.15.136" 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: mon election default strategy = 3 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug inject dispatch delay duration = 0.1 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug inject dispatch delay probability = 0.1 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: osd_async_recovery_min_cost = 1 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: fsid = b8d0ffd9-6668-442b-a039-a24cea734f3e 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout:[osd] 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: osd journal size = 100 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub load threshold = 5.0 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub max interval = 120 2024-04-18T23:09:18.279 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock profile = high_recovery_ops 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd recover clone overlap = true 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd recovery max chunk = 1048576 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug shutdown = true 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug op order = true 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify stray on activate = true 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug trim objects = true 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd open classes on start = true 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug pg log writeout = true 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd deep scrub update digest min age = 30 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.280 INFO:teuthology.orchestra.run.smithi142.stdout: osd map max advance = 10 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: journal zero on create = true 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: filestore ondisk finisher threads = 3 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: filestore apply finisher threads = 3 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: bdev debug aio = true 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug misdirected ops = true 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: debug monc = 20 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: bdev async discard = True 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: bdev enable discard = True 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore allocator = bitmap 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore block size = 96636764160 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore fsck on mount = True 2024-04-18T23:09:18.281 INFO:teuthology.orchestra.run.smithi142.stdout: bluestore zero block detection = True 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluefs = 20 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: debug bluestore = 20 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: debug osd = 20 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: debug rocksdb = 10 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd backfillfull_ratio = 0.85 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd full ratio = 0.9 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd nearfull ratio = 0.8 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: osd blocked scrub grace period = 3600 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug reject backfill probability = 0.3 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify cached snaps = True 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: osd debug verify missing on start = True 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: osd failsafe full ratio = 0.95 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: osd max backfills = 3 2024-04-18T23:09:18.282 INFO:teuthology.orchestra.run.smithi142.stdout: osd max markdown count = 1000 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: osd mclock override recovery settings = True 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: osd objectstore = bluestore 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: osd op queue = debug_random 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: osd op queue cut off = debug_random 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: osd scrub min interval = 60 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: osd shutdown pgref assert = True 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout:[mgr] 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: debug mgr = 20 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: mgr/telemetry/nag = false 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.283 INFO:teuthology.orchestra.run.smithi142.stdout:[mon] 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: debug ms = 1 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: debug mon = 20 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: debug paxos = 20 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: debug auth = 20 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: mon data avail warn = 5 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: mon mgr mkfs grace = 240 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min pgs per osd = 4 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd reporter subtree level = osd 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: mon osd prime pg temp = true 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: mon reweight min bytes per osd = 10 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: auth mon ticket ttl = 660 # 11m 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: auth service ticket ttl = 240 # 4m 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.284 INFO:teuthology.orchestra.run.smithi142.stdout: # don't complain about insecure global_id in the test suite 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: # 1m isn't quite enough 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: mon_down_mkfs_grace = 2m 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: mon_warn_on_filestore_osds = false 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: mon scrub interval = 300 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout:[client] 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: rgw cache enabled = true 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable ops log = true 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: rgw enable usage log = true 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-18T23:09:18.285 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.a] 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.b] 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi142.stdout:[mon.c] 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout:[global] 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: chdir = "" 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: auth supported = cephx 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: filestore xattr use omap = true 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: mon clock drift allowed = 1.000 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: osd crush chooseleaf type = 0 2024-04-18T23:09:18.286 INFO:teuthology.orchestra.run.smithi136.stdout: auth debug = true 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on old message = true 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: ms die on bug = true 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: mon pg warn max object skew = 0 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: # disable pg_autoscaler by default for new pools 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default size = 2 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow primary affinity = true 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd allow pg remap = true 2024-04-18T23:09:18.287 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on legacy crush tunables = false 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on crush straw calc version zero = false 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on no sortbitwise = false 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on osd down out interval zero = false 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: mon warn on too few osds = false 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: mon_allow_pool_size_one = true 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: osd default data pool replay window = 5 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: mon allow pool delete = true 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.288 INFO:teuthology.orchestra.run.smithi136.stdout: mon cluster log file level = debug 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: debug asserts on shutdown = true 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: mon health detail to clog = false 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: mon host = "172.21.15.73,172.21.15.120,172.21.15.136" 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: mon election default strategy = 3 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug inject dispatch delay duration = 0.1 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug inject dispatch delay probability = 0.1 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: osd_async_recovery_min_cost = 1 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: fsid = b8d0ffd9-6668-442b-a039-a24cea734f3e 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout:[osd] 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: osd journal size = 100 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub load threshold = 5.0 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub max interval = 120 2024-04-18T23:09:18.289 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock profile = high_recovery_ops 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd recover clone overlap = true 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd recovery max chunk = 1048576 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug shutdown = true 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug op order = true 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify stray on activate = true 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug trim objects = true 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd open classes on start = true 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug pg log writeout = true 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd deep scrub update digest min age = 30 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.290 INFO:teuthology.orchestra.run.smithi136.stdout: osd map max advance = 10 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: journal zero on create = true 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: filestore ondisk finisher threads = 3 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: filestore apply finisher threads = 3 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: bdev debug aio = true 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug misdirected ops = true 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: debug monc = 20 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: bdev async discard = True 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: bdev enable discard = True 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore allocator = bitmap 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore block size = 96636764160 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore fsck on mount = True 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: bluestore zero block detection = True 2024-04-18T23:09:18.291 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluefs = 20 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: debug bluestore = 20 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: debug osd = 20 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: debug rocksdb = 10 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd backfillfull_ratio = 0.85 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd full ratio = 0.9 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd nearfull ratio = 0.8 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd blocked scrub grace period = 3600 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug reject backfill probability = 0.3 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify cached snaps = True 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd debug verify missing on start = True 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd failsafe full ratio = 0.95 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd max backfills = 3 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd max markdown count = 1000 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd mclock override recovery settings = True 2024-04-18T23:09:18.292 INFO:teuthology.orchestra.run.smithi136.stdout: osd objectstore = bluestore 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: osd op queue = debug_random 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: osd op queue cut off = debug_random 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: osd scrub min interval = 60 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: osd shutdown pgref assert = True 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr] 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: debug mgr = 20 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: mgr/telemetry/nag = false 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout:[mon] 2024-04-18T23:09:18.293 INFO:teuthology.orchestra.run.smithi136.stdout: debug ms = 1 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: debug mon = 20 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: debug paxos = 20 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: debug auth = 20 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: mon data avail warn = 5 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: mon mgr mkfs grace = 240 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min pgs per osd = 4 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd reporter subtree level = osd 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: mon osd prime pg temp = true 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: mon reweight min bytes per osd = 10 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: auth mon ticket ttl = 660 # 11m 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: auth service ticket ttl = 240 # 4m 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.294 INFO:teuthology.orchestra.run.smithi136.stdout: # don't complain about insecure global_id in the test suite 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: # 1m isn't quite enough 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: mon_down_mkfs_grace = 2m 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: mon_warn_on_filestore_osds = false 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: mon scrub interval = 300 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout:[client] 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: rgw cache enabled = true 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable ops log = true 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: rgw enable usage log = true 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-18T23:09:18.295 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.a] 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.b] 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi136.stdout:[mon.c] 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout:[global] 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: chdir = "" 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: auth supported = cephx 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: filestore xattr use omap = true 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: mon clock drift allowed = 1.000 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: osd crush chooseleaf type = 0 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: auth debug = true 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.296 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on old message = true 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: ms die on bug = true 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: mon pg warn max object skew = 0 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: # disable pg_autoscaler by default for new pools 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default size = 2 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow primary affinity = true 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd allow pg remap = true 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on legacy crush tunables = false 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on crush straw calc version zero = false 2024-04-18T23:09:18.297 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on no sortbitwise = false 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on osd down out interval zero = false 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: mon warn on too few osds = false 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: mon_allow_pool_size_one = true 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: osd default data pool replay window = 5 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: mon allow pool delete = true 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: mon cluster log file level = debug 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: debug asserts on shutdown = true 2024-04-18T23:09:18.298 INFO:teuthology.orchestra.run.smithi120.stdout: mon health detail to clog = false 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: mon host = "172.21.15.73,172.21.15.120,172.21.15.136" 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: mon election default strategy = 3 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug inject dispatch delay duration = 0.1 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug inject dispatch delay probability = 0.1 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: osd_async_recovery_min_cost = 1 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: fsid = b8d0ffd9-6668-442b-a039-a24cea734f3e 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout:[osd] 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: osd journal size = 100 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub load threshold = 5.0 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub max interval = 120 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: osd mclock profile = high_recovery_ops 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.299 INFO:teuthology.orchestra.run.smithi120.stdout: osd recover clone overlap = true 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: osd recovery max chunk = 1048576 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug shutdown = true 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug op order = true 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug verify stray on activate = true 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug trim objects = true 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: osd open classes on start = true 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug pg log writeout = true 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: osd deep scrub update digest min age = 30 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: osd map max advance = 10 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.300 INFO:teuthology.orchestra.run.smithi120.stdout: journal zero on create = true 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: filestore ondisk finisher threads = 3 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: filestore apply finisher threads = 3 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: bdev debug aio = true 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug misdirected ops = true 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: debug monc = 20 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: bdev async discard = True 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: bdev enable discard = True 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore allocator = bitmap 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore block size = 96636764160 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore fsck on mount = True 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: bluestore zero block detection = True 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluefs = 20 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: debug bluestore = 20 2024-04-18T23:09:18.301 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: debug osd = 20 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: debug rocksdb = 10 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd backfillfull_ratio = 0.85 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd full ratio = 0.9 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd nearfull ratio = 0.8 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: osd blocked scrub grace period = 3600 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug reject backfill probability = 0.3 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug verify cached snaps = True 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: osd debug verify missing on start = True 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: osd failsafe full ratio = 0.95 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: osd max backfills = 3 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: osd max markdown count = 1000 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: osd mclock override recovery settings = True 2024-04-18T23:09:18.302 INFO:teuthology.orchestra.run.smithi120.stdout: osd objectstore = bluestore 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: osd op queue = debug_random 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: osd op queue cut off = debug_random 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: osd scrub min interval = 60 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: osd shutdown pgref assert = True 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr] 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: debug mgr = 20 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: mgr/telemetry/nag = false 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout:[mon] 2024-04-18T23:09:18.303 INFO:teuthology.orchestra.run.smithi120.stdout: debug ms = 1 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: debug mon = 20 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: debug paxos = 20 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: debug auth = 20 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: mon data avail warn = 5 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: mon mgr mkfs grace = 240 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min pgs per osd = 4 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd reporter subtree level = osd 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: mon osd prime pg temp = true 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: mon reweight min bytes per osd = 10 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: auth mon ticket ttl = 660 # 11m 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: auth service ticket ttl = 240 # 4m 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: # don't complain about insecure global_id in the test suite 2024-04-18T23:09:18.304 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: # 1m isn't quite enough 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: mon_down_mkfs_grace = 2m 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: mon_warn_on_filestore_osds = false 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: mon scrub interval = 300 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout:[client] 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: rgw cache enabled = true 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable ops log = true 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: rgw enable usage log = true 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-18T23:09:18.305 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.a] 2024-04-18T23:09:18.306 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.b] 2024-04-18T23:09:18.306 INFO:teuthology.orchestra.run.smithi120.stdout:[mon.c] 2024-04-18T23:09:18.308 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-18T23:09:18.308 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-18T23:09:18.401 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-18T23:09:18.401 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:18.401 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-18T23:09:18.407 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:18.407 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-18T23:09:18.452 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:09:18.452 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:18.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-18T23:09:18.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T23:09:18.516 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:09:18.516 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:18.564 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:09:18.564 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:09:18.564 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-18T23:09:18.564 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T23:09:18.586 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:09:18.586 DEBUG:teuthology.orchestra.run.smithi120:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:18.634 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:09:18.634 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:09:18.634 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-18T23:09:18.634 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T23:09:18.657 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:09:18.657 DEBUG:teuthology.orchestra.run.smithi136:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:18.703 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:09:18.703 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:09:18.703 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-18T23:09:18.703 DEBUG:teuthology.orchestra.run.smithi142:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-18T23:09:18.726 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:09:18.726 DEBUG:teuthology.orchestra.run.smithi142:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:18.773 INFO:tasks.ceph:Setting up mon nodes... 2024-04-18T23:09:18.774 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-18T23:09:18.774 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-18T23:09:18.817 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-18T23:09:18.820 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-18T23:09:18.862 INFO:teuthology.orchestra.run.smithi142.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-18T23:09:18.879 INFO:tasks.ceph:Setting up mds nodes... 2024-04-18T23:09:18.879 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-18T23:09:18.879 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-18T23:09:18.955 INFO:teuthology.orchestra.run.smithi142.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-18T23:09:18.967 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-18T23:09:18.967 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi120.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi136.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi142.front.sepia.ceph.com'): {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'}} 2024-04-18T23:09:18.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-18T23:09:18.980 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:18.980 INFO:tasks.ceph:role: osd.0 2024-04-18T23:09:18.981 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:09:18.981 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-18T23:09:19.043 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:19.043 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:19.043 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:19.044 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:19.044 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:19.044 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:19.044 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:19.044 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:19.044 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:19.044 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:19.490 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-18T23:09:19.492 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-18T23:09:19.492 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-18T23:09:19.589 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-18T23:09:19.598 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:19.598 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:19.599 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-18T23:09:19.649 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:19.649 INFO:tasks.ceph:role: osd.4 2024-04-18T23:09:19.649 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:09:19.649 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-18T23:09:19.713 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:19.714 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:19.714 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:19.714 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:19.714 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:19.714 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:20.178 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-18T23:09:20.180 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-18T23:09:20.180 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-4 2024-04-18T23:09:20.335 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-18T23:09:20.345 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:20.346 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:20.346 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-8 2024-04-18T23:09:20.398 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:20.399 INFO:tasks.ceph:role: osd.8 2024-04-18T23:09:20.399 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:09:20.399 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-18T23:09:20.458 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:20.458 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:20.458 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:20.458 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:20.458 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:20.459 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:20.927 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-18T23:09:20.930 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-18T23:09:20.930 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-8 2024-04-18T23:09:21.066 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-8 2024-04-18T23:09:21.077 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:21.078 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:21.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-12 2024-04-18T23:09:21.131 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_2', 'osd.8': '/dev/vg_nvme/lv_3', 'osd.12': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:21.131 INFO:tasks.ceph:role: osd.12 2024-04-18T23:09:21.131 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:09:21.131 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-18T23:09:21.195 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:21.195 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:21.195 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:21.196 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:21.196 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:21.196 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:21.196 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:21.196 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:21.196 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:21.196 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:21.626 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2024-04-18T23:09:21.629 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2024-04-18T23:09:21.629 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-12 2024-04-18T23:09:21.715 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-12 2024-04-18T23:09:21.769 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:21.769 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:21.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:21.886 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:21.883+0000 7ff890c4e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-18T23:09:21.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:21.883+0000 7ff890c4e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-18T23:09:21.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:21.883+0000 7ff890c4e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-18T23:09:21.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:21.883+0000 7ff890c4e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-18T23:09:21.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:21.883+0000 7ff890c4e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-18T23:09:21.887 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:21.883+0000 7ff890c4e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-18T23:09:24.876 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-18T23:09:24.890 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:24.971+0000 7f4267c087c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-18T23:09:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:24.971+0000 7f4267c087c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-18T23:09:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:24.971+0000 7f4267c087c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-18T23:09:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:24.971+0000 7f4267c087c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-18T23:09:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:24.971+0000 7f4267c087c0 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-18T23:09:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:24.971+0000 7f4267c087c0 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-18T23:09:27.952 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-18T23:09:28.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 8 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:28.095 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:28.091+0000 7f8e7576d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-8/keyring: can't open /var/lib/ceph/osd/ceph-8/keyring: (2) No such file or directory 2024-04-18T23:09:28.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:28.095+0000 7f8e7576d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-8/keyring 2024-04-18T23:09:28.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:28.095+0000 7f8e7576d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-18T23:09:28.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:28.095+0000 7f8e7576d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-18T23:09:28.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:28.095+0000 7f8e7576d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-8/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-8/block: (2) No such file or directory 2024-04-18T23:09:28.096 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:28.095+0000 7f8e7576d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-8) _read_fsid unparsable uuid 2024-04-18T23:09:31.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-8 2024-04-18T23:09:31.120 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 12 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:31.203+0000 7eff3c6ee7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-12/keyring: can't open /var/lib/ceph/osd/ceph-12/keyring: (2) No such file or directory 2024-04-18T23:09:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:31.203+0000 7eff3c6ee7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-12/keyring 2024-04-18T23:09:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:31.203+0000 7eff3c6ee7c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-18T23:09:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:31.203+0000 7eff3c6ee7c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-18T23:09:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:31.203+0000 7eff3c6ee7c0 -1 bluestore(/var/lib/ceph/osd/ceph-12/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-12/block: (2) No such file or directory 2024-04-18T23:09:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2024-04-18T23:09:31.203+0000 7eff3c6ee7c0 -1 bluestore(/var/lib/ceph/osd/ceph-12) _read_fsid unparsable uuid 2024-04-18T23:09:34.171 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-12 2024-04-18T23:09:34.229 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-18T23:09:34.243 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:34.243 INFO:tasks.ceph:role: osd.1 2024-04-18T23:09:34.243 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:09:34.243 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-18T23:09:34.305 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:34.306 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:34.306 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:34.306 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:34.306 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:34.306 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:34.765 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-04-18T23:09:34.767 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-04-18T23:09:34.767 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-18T23:09:34.859 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-18T23:09:34.868 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:34.869 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:34.869 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-18T23:09:34.927 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:34.927 INFO:tasks.ceph:role: osd.5 2024-04-18T23:09:34.927 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:09:34.927 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-18T23:09:34.991 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:34.991 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:34.991 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:34.992 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:34.992 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:34.992 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:34.992 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:35.467 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-04-18T23:09:35.470 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-04-18T23:09:35.470 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-18T23:09:35.640 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-18T23:09:35.652 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:35.652 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:35.652 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-9 2024-04-18T23:09:35.705 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:35.705 INFO:tasks.ceph:role: osd.9 2024-04-18T23:09:35.705 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:09:35.705 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:35.770 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:36.249 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-04-18T23:09:36.251 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-04-18T23:09:36.251 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-9 2024-04-18T23:09:36.375 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-9 2024-04-18T23:09:36.386 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:36.387 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:36.387 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/osd/ceph-13 2024-04-18T23:09:36.441 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.9': '/dev/vg_nvme/lv_3', 'osd.13': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:36.441 INFO:tasks.ceph:role: osd.13 2024-04-18T23:09:36.441 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:09:36.441 DEBUG:teuthology.orchestra.run.smithi120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:36.505 INFO:teuthology.orchestra.run.smithi120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:36.927 INFO:teuthology.orchestra.run.smithi120.stdout:Discarding blocks...Done. 2024-04-18T23:09:36.929 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi120.front.sepia.ceph.com -o noatime 2024-04-18T23:09:36.929 DEBUG:teuthology.orchestra.run.smithi120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-13 2024-04-18T23:09:37.034 DEBUG:teuthology.orchestra.run.smithi120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-13 2024-04-18T23:09:37.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:37.044 INFO:teuthology.orchestra.run.smithi120.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:37.044 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:37.167 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:37.168+0000 7f9956f1d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-18T23:09:37.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:37.172+0000 7f9956f1d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-18T23:09:37.170 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:37.172+0000 7f9956f1d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-18T23:09:37.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:37.172+0000 7f9956f1d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-18T23:09:37.171 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:37.172+0000 7f9956f1d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-18T23:09:37.173 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:37.172+0000 7f9956f1d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-18T23:09:40.142 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-18T23:09:40.156 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:40.234 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:40.235+0000 7f049c2017c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-18T23:09:40.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:40.235+0000 7f049c2017c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-18T23:09:40.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:40.235+0000 7f049c2017c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-18T23:09:40.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:40.235+0000 7f049c2017c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-18T23:09:40.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:40.235+0000 7f049c2017c0 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-18T23:09:40.236 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:40.235+0000 7f049c2017c0 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-18T23:09:43.208 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-18T23:09:43.262 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 9 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:43.345 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:43.343+0000 7f1008d967c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-9/keyring: can't open /var/lib/ceph/osd/ceph-9/keyring: (2) No such file or directory 2024-04-18T23:09:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:43.347+0000 7f1008d967c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-9/keyring 2024-04-18T23:09:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:43.347+0000 7f1008d967c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-18T23:09:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:43.347+0000 7f1008d967c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-18T23:09:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:43.347+0000 7f1008d967c0 -1 bluestore(/var/lib/ceph/osd/ceph-9/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-9/block: (2) No such file or directory 2024-04-18T23:09:43.346 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:43.347+0000 7f1008d967c0 -1 bluestore(/var/lib/ceph/osd/ceph-9) _read_fsid unparsable uuid 2024-04-18T23:09:46.332 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-9 2024-04-18T23:09:46.384 DEBUG:teuthology.orchestra.run.smithi120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 13 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:46.468 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:46.467+0000 7f37bc19c7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-13/keyring: can't open /var/lib/ceph/osd/ceph-13/keyring: (2) No such file or directory 2024-04-18T23:09:46.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:46.467+0000 7f37bc19c7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-13/keyring 2024-04-18T23:09:46.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:46.467+0000 7f37bc19c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-18T23:09:46.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:46.467+0000 7f37bc19c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-18T23:09:46.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:46.467+0000 7f37bc19c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-13/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-13/block: (2) No such file or directory 2024-04-18T23:09:46.469 INFO:teuthology.orchestra.run.smithi120.stderr:2024-04-18T23:09:46.471+0000 7f37bc19c7c0 -1 bluestore(/var/lib/ceph/osd/ceph-13) _read_fsid unparsable uuid 2024-04-18T23:09:49.440 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-13 2024-04-18T23:09:49.492 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-18T23:09:49.507 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:49.507 INFO:tasks.ceph:role: osd.2 2024-04-18T23:09:49.507 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:09:49.507 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-18T23:09:49.569 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:49.570 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:49.570 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:49.570 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:49.570 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:50.063 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-04-18T23:09:50.065 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-04-18T23:09:50.065 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2024-04-18T23:09:50.170 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-18T23:09:50.181 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:50.181 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:50.182 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-18T23:09:50.237 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:50.237 INFO:tasks.ceph:role: osd.6 2024-04-18T23:09:50.237 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:09:50.237 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:50.299 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:50.718 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-04-18T23:09:50.720 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-04-18T23:09:50.720 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2024-04-18T23:09:50.780 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-18T23:09:50.830 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:50.830 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:50.830 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-10 2024-04-18T23:09:50.883 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:50.883 INFO:tasks.ceph:role: osd.10 2024-04-18T23:09:50.883 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:09:50.883 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:50.946 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:51.422 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-04-18T23:09:51.425 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-04-18T23:09:51.425 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-10 2024-04-18T23:09:51.547 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-10 2024-04-18T23:09:51.559 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:51.559 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:51.559 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/osd/ceph-14 2024-04-18T23:09:51.613 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.6': '/dev/vg_nvme/lv_2', 'osd.10': '/dev/vg_nvme/lv_3', 'osd.14': '/dev/vg_nvme/lv_4'} 2024-04-18T23:09:51.614 INFO:tasks.ceph:role: osd.14 2024-04-18T23:09:51.614 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:09:51.614 DEBUG:teuthology.orchestra.run.smithi136:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-18T23:09:51.678 INFO:teuthology.orchestra.run.smithi136.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:09:51.679 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:09:51.679 INFO:teuthology.orchestra.run.smithi136.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:09:51.679 INFO:teuthology.orchestra.run.smithi136.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:09:51.679 INFO:teuthology.orchestra.run.smithi136.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:09:51.679 INFO:teuthology.orchestra.run.smithi136.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:09:51.679 INFO:teuthology.orchestra.run.smithi136.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:09:51.679 INFO:teuthology.orchestra.run.smithi136.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:09:51.679 INFO:teuthology.orchestra.run.smithi136.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:09:51.679 INFO:teuthology.orchestra.run.smithi136.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:09:52.178 INFO:teuthology.orchestra.run.smithi136.stdout:Discarding blocks...Done. 2024-04-18T23:09:52.180 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi136.front.sepia.ceph.com -o noatime 2024-04-18T23:09:52.180 DEBUG:teuthology.orchestra.run.smithi136:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-14 2024-04-18T23:09:52.291 DEBUG:teuthology.orchestra.run.smithi136:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-14 2024-04-18T23:09:52.303 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:09:52.303 INFO:teuthology.orchestra.run.smithi136.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:09:52.303 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:52.695 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:52.697+0000 7fdbe995e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-18T23:09:52.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:52.701+0000 7fdbe995e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-18T23:09:52.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:52.701+0000 7fdbe995e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-18T23:09:52.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:52.701+0000 7fdbe995e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-18T23:09:52.700 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:52.701+0000 7fdbe995e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-18T23:09:52.703 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:52.705+0000 7fdbe995e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-18T23:09:55.703 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-18T23:09:55.718 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:55.803 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:55.805+0000 7fb31b5637c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-18T23:09:55.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:55.805+0000 7fb31b5637c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-18T23:09:55.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:55.805+0000 7fb31b5637c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-18T23:09:55.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:55.805+0000 7fb31b5637c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-18T23:09:55.804 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:55.805+0000 7fb31b5637c0 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-18T23:09:55.805 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:55.805+0000 7fb31b5637c0 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-18T23:09:58.779 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-18T23:09:58.837 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 10 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:09:58.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:58.925+0000 7f8a4770d7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-10/keyring: can't open /var/lib/ceph/osd/ceph-10/keyring: (2) No such file or directory 2024-04-18T23:09:58.922 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:58.925+0000 7f8a4770d7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-10/keyring 2024-04-18T23:09:58.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:58.925+0000 7f8a4770d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-18T23:09:58.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:58.925+0000 7f8a4770d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-18T23:09:58.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:58.925+0000 7f8a4770d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-10/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-10/block: (2) No such file or directory 2024-04-18T23:09:58.923 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:09:58.925+0000 7f8a4770d7c0 -1 bluestore(/var/lib/ceph/osd/ceph-10) _read_fsid unparsable uuid 2024-04-18T23:10:01.900 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-10 2024-04-18T23:10:01.955 DEBUG:teuthology.orchestra.run.smithi136:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 14 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:10:02.036 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:10:02.037+0000 7fdf7d4667c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-14/keyring: can't open /var/lib/ceph/osd/ceph-14/keyring: (2) No such file or directory 2024-04-18T23:10:02.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:10:02.037+0000 7fdf7d4667c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-14/keyring 2024-04-18T23:10:02.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:10:02.037+0000 7fdf7d4667c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-18T23:10:02.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:10:02.037+0000 7fdf7d4667c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-18T23:10:02.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:10:02.037+0000 7fdf7d4667c0 -1 bluestore(/var/lib/ceph/osd/ceph-14/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-14/block: (2) No such file or directory 2024-04-18T23:10:02.037 INFO:teuthology.orchestra.run.smithi136.stderr:2024-04-18T23:10:02.037+0000 7fdf7d4667c0 -1 bluestore(/var/lib/ceph/osd/ceph-14) _read_fsid unparsable uuid 2024-04-18T23:10:05.020 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-14 2024-04-18T23:10:05.074 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-18T23:10:05.088 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-18T23:10:05.088 INFO:tasks.ceph:role: osd.3 2024-04-18T23:10:05.088 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:10:05.088 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:10:05.152 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:10:05.604 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-04-18T23:10:05.607 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-04-18T23:10:05.607 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-3 2024-04-18T23:10:05.700 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-18T23:10:05.709 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:10:05.710 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:10:05.710 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-18T23:10:05.761 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-18T23:10:05.762 INFO:tasks.ceph:role: osd.7 2024-04-18T23:10:05.762 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:10:05.762 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-18T23:10:05.824 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:10:05.824 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:10:05.824 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:10:05.824 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:10:05.824 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:10:05.824 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:10:05.825 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:10:05.825 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:10:05.825 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:10:05.825 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:10:06.302 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-04-18T23:10:06.303 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-04-18T23:10:06.304 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-7 2024-04-18T23:10:06.424 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-18T23:10:06.435 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:10:06.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:10:06.436 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-11 2024-04-18T23:10:06.492 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-18T23:10:06.492 INFO:tasks.ceph:role: osd.11 2024-04-18T23:10:06.492 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:10:06.492 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-18T23:10:06.557 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:10:06.557 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:10:06.557 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:10:06.557 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:10:06.557 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:10:06.558 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:10:06.558 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:10:06.558 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:10:06.558 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:10:06.558 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:10:07.041 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-04-18T23:10:07.043 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-04-18T23:10:07.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-11 2024-04-18T23:10:07.206 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-11 2024-04-18T23:10:07.217 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:10:07.218 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:10:07.218 DEBUG:teuthology.orchestra.run.smithi142:> sudo mkdir -p /var/lib/ceph/osd/ceph-15 2024-04-18T23:10:07.272 INFO:tasks.ceph:roles_to_devs: {'osd.3': '/dev/vg_nvme/lv_1', 'osd.7': '/dev/vg_nvme/lv_2', 'osd.11': '/dev/vg_nvme/lv_3', 'osd.15': '/dev/vg_nvme/lv_4'} 2024-04-18T23:10:07.272 INFO:tasks.ceph:role: osd.15 2024-04-18T23:10:07.273 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:10:07.273 DEBUG:teuthology.orchestra.run.smithi142:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-18T23:10:07.340 INFO:teuthology.orchestra.run.smithi142.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-18T23:10:07.340 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-18T23:10:07.340 INFO:teuthology.orchestra.run.smithi142.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-18T23:10:07.340 INFO:teuthology.orchestra.run.smithi142.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-18T23:10:07.341 INFO:teuthology.orchestra.run.smithi142.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-18T23:10:07.341 INFO:teuthology.orchestra.run.smithi142.stdout: = sunit=0 swidth=0 blks 2024-04-18T23:10:07.341 INFO:teuthology.orchestra.run.smithi142.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-18T23:10:07.341 INFO:teuthology.orchestra.run.smithi142.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-18T23:10:07.341 INFO:teuthology.orchestra.run.smithi142.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-18T23:10:07.341 INFO:teuthology.orchestra.run.smithi142.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-18T23:10:07.767 INFO:teuthology.orchestra.run.smithi142.stdout:Discarding blocks...Done. 2024-04-18T23:10:07.769 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi142.front.sepia.ceph.com -o noatime 2024-04-18T23:10:07.769 DEBUG:teuthology.orchestra.run.smithi142:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-15 2024-04-18T23:10:07.865 DEBUG:teuthology.orchestra.run.smithi142:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-15 2024-04-18T23:10:07.874 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:10:07.875 INFO:teuthology.orchestra.run.smithi142.stderr:sudo: /sbin/restorecon: command not found 2024-04-18T23:10:07.875 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:10:08.092 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:08.095+0000 7fea6d56e7c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-18T23:10:08.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:08.099+0000 7fea6d56e7c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-18T23:10:08.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:08.099+0000 7fea6d56e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-18T23:10:08.096 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:08.099+0000 7fea6d56e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-18T23:10:08.097 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:08.099+0000 7fea6d56e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-18T23:10:08.098 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:08.099+0000 7fea6d56e7c0 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-18T23:10:11.073 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-18T23:10:11.088 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:10:11.169 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:11.171+0000 7fbde95f97c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-18T23:10:11.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:11.171+0000 7fbde95f97c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-18T23:10:11.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:11.171+0000 7fbde95f97c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-18T23:10:11.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:11.171+0000 7fbde95f97c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-18T23:10:11.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:11.171+0000 7fbde95f97c0 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-18T23:10:11.170 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:11.171+0000 7fbde95f97c0 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-18T23:10:14.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-18T23:10:14.213 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 11 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:10:14.299 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:14.299+0000 7f97a6e567c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-11/keyring: can't open /var/lib/ceph/osd/ceph-11/keyring: (2) No such file or directory 2024-04-18T23:10:14.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:14.299+0000 7f97a6e567c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-11/keyring 2024-04-18T23:10:14.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:14.299+0000 7f97a6e567c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-18T23:10:14.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:14.299+0000 7f97a6e567c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-18T23:10:14.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:14.299+0000 7f97a6e567c0 -1 bluestore(/var/lib/ceph/osd/ceph-11/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-11/block: (2) No such file or directory 2024-04-18T23:10:14.300 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:14.303+0000 7f97a6e567c0 -1 bluestore(/var/lib/ceph/osd/ceph-11) _read_fsid unparsable uuid 2024-04-18T23:10:17.267 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-11 2024-04-18T23:10:17.321 DEBUG:teuthology.orchestra.run.smithi142:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 15 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:10:17.402 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:17.403+0000 7eff296547c0 -1 auth: error reading file: /var/lib/ceph/osd/ceph-15/keyring: can't open /var/lib/ceph/osd/ceph-15/keyring: (2) No such file or directory 2024-04-18T23:10:17.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:17.403+0000 7eff296547c0 -1 created new key in keyring /var/lib/ceph/osd/ceph-15/keyring 2024-04-18T23:10:17.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:17.403+0000 7eff296547c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-18T23:10:17.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:17.403+0000 7eff296547c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-18T23:10:17.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:17.403+0000 7eff296547c0 -1 bluestore(/var/lib/ceph/osd/ceph-15/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-15/block: (2) No such file or directory 2024-04-18T23:10:17.403 INFO:teuthology.orchestra.run.smithi142.stderr:2024-04-18T23:10:17.403+0000 7eff296547c0 -1 bluestore(/var/lib/ceph/osd/ceph-15) _read_fsid unparsable uuid 2024-04-18T23:10:20.404 DEBUG:teuthology.orchestra.run.smithi142:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-15 2024-04-18T23:10:20.465 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-18T23:10:20.465 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:10:20.465 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-18T23:10:20.479 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:10:20.480 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-18T23:10:20.537 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:10:20.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-18T23:10:20.591 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:10:20.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-8/keyring of=/dev/stdout 2024-04-18T23:10:20.647 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:10:20.647 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-12/keyring of=/dev/stdout 2024-04-18T23:10:20.700 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:10:20.700 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-18T23:10:20.714 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:10:20.714 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-18T23:10:20.769 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:10:20.769 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-9/keyring of=/dev/stdout 2024-04-18T23:10:20.824 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:10:20.824 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-13/keyring of=/dev/stdout 2024-04-18T23:10:20.880 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:10:20.880 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-18T23:10:20.893 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:10:20.894 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-18T23:10:20.948 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:10:20.948 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-10/keyring of=/dev/stdout 2024-04-18T23:10:21.000 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:10:21.000 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-14/keyring of=/dev/stdout 2024-04-18T23:10:21.053 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:21.053 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-18T23:10:21.066 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:21.066 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-18T23:10:21.119 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:21.119 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-18T23:10:21.171 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:21.171 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-11/keyring of=/dev/stdout 2024-04-18T23:10:21.227 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:21.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-15/keyring of=/dev/stdout 2024-04-18T23:10:21.279 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:21.279 DEBUG:teuthology.orchestra.run.smithi142:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-18T23:10:21.324 INFO:tasks.ceph:Adding keys to all mons... 2024-04-18T23:10:21.324 DEBUG:teuthology.orchestra.run.smithi073:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-18T23:10:21.327 DEBUG:teuthology.orchestra.run.smithi120:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-18T23:10:21.329 DEBUG:teuthology.orchestra.run.smithi136:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-18T23:10:21.336 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.y] 2024-04-18T23:10:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAeqCFmBDHGMBAAKwk00n1RzIa75/MUv1pVEg== 2024-04-18T23:10:21.336 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.0] 2024-04-18T23:10:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAhqCFmZG/cNBAAvrm3Dqbd5ogQgH1ubzd+yQ== 2024-04-18T23:10:21.336 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.4] 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAkqCFmfPghOhAA1u10705aCcG60mm8TmLlVQ== 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.8] 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAoqCFmOv3BBRAA4sXqhcvUtlmiTj8hmrOcCg== 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.12] 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQArqCFmIMI6DBAApOxPbQuEnQbf87t2vp1M1Q== 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.1] 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAxqCFmjtc5ChAAa03o9TQtyIIK8Rv4KhLWew== 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.5] 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA0qCFmKMw3DhAA3HFi/yBJeJvdiQBH+4nxEg== 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.9] 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA3qCFmhj3PFBAA9hMOjI3PHVHsdRjO5vwyig== 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.13] 2024-04-18T23:10:21.337 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQA6qCFmlZEpHBAAbYBdpEmkjmB00C9g0MuL5w== 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.2] 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBAqCFmFgWuKRAAOYjCWO62tr9uzI5DB/S95Q== 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.6] 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBDqCFmhM8kMBAAAOGKKgm114/TEYmTNfoy1g== 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.10] 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBGqCFmGuowNxAAdGTESMJoBfJ/MhDOYAG1zw== 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.14] 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBKqCFmCxJkAhAAyvpJSdhleTwpkPRhKUl1Lg== 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.x] 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAeqCFmrDyXMxAAzAZxP1AaDc06fT8t0rMebg== 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.3] 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBQqCFmYLq3BRAAJwYAF0c/kWce7BO51/IKzw== 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.7] 2024-04-18T23:10:21.338 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBTqCFmCU5SChAAXRdjDM5S+9ZmdNONLkGczA== 2024-04-18T23:10:21.339 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.11] 2024-04-18T23:10:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBWqCFmueQOEhAAu+ktUlfQzEgzFqQXoRk5Ag== 2024-04-18T23:10:21.339 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.15] 2024-04-18T23:10:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQBZqCFm0tQwGBAAlKKhO9HVACxvK1RxBy9a8Q== 2024-04-18T23:10:21.339 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2024-04-18T23:10:21.339 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQAeqCFm+x4ZORAA10dLPE537IDkfPfQ/2CBjQ== 2024-04-18T23:10:21.339 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.y] 2024-04-18T23:10:21.339 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQAeqCFmBDHGMBAAKwk00n1RzIa75/MUv1pVEg== 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.0] 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQAhqCFmZG/cNBAAvrm3Dqbd5ogQgH1ubzd+yQ== 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.4] 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQAkqCFmfPghOhAA1u10705aCcG60mm8TmLlVQ== 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.8] 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQAoqCFmOv3BBRAA4sXqhcvUtlmiTj8hmrOcCg== 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.12] 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQArqCFmIMI6DBAApOxPbQuEnQbf87t2vp1M1Q== 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.1] 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQAxqCFmjtc5ChAAa03o9TQtyIIK8Rv4KhLWew== 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.5] 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA0qCFmKMw3DhAA3HFi/yBJeJvdiQBH+4nxEg== 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.9] 2024-04-18T23:10:21.340 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA3qCFmhj3PFBAA9hMOjI3PHVHsdRjO5vwyig== 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.13] 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQA6qCFmlZEpHBAAbYBdpEmkjmB00C9g0MuL5w== 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.2] 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBAqCFmFgWuKRAAOYjCWO62tr9uzI5DB/S95Q== 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.6] 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBDqCFmhM8kMBAAAOGKKgm114/TEYmTNfoy1g== 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.10] 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBGqCFmGuowNxAAdGTESMJoBfJ/MhDOYAG1zw== 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.14] 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBKqCFmCxJkAhAAyvpJSdhleTwpkPRhKUl1Lg== 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout:[mgr.x] 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQAeqCFmrDyXMxAAzAZxP1AaDc06fT8t0rMebg== 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.3] 2024-04-18T23:10:21.341 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBQqCFmYLq3BRAAJwYAF0c/kWce7BO51/IKzw== 2024-04-18T23:10:21.342 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.7] 2024-04-18T23:10:21.342 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBTqCFmCU5SChAAXRdjDM5S+9ZmdNONLkGczA== 2024-04-18T23:10:21.342 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.11] 2024-04-18T23:10:21.342 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBWqCFmueQOEhAAu+ktUlfQzEgzFqQXoRk5Ag== 2024-04-18T23:10:21.342 INFO:teuthology.orchestra.run.smithi120.stdout:[osd.15] 2024-04-18T23:10:21.342 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQBZqCFm0tQwGBAAlKKhO9HVACxvK1RxBy9a8Q== 2024-04-18T23:10:21.342 INFO:teuthology.orchestra.run.smithi120.stdout:[client.0] 2024-04-18T23:10:21.342 INFO:teuthology.orchestra.run.smithi120.stdout: key = AQAeqCFm+x4ZORAA10dLPE537IDkfPfQ/2CBjQ== 2024-04-18T23:10:21.344 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.y] 2024-04-18T23:10:21.344 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAeqCFmBDHGMBAAKwk00n1RzIa75/MUv1pVEg== 2024-04-18T23:10:21.344 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.0] 2024-04-18T23:10:21.344 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAhqCFmZG/cNBAAvrm3Dqbd5ogQgH1ubzd+yQ== 2024-04-18T23:10:21.344 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.4] 2024-04-18T23:10:21.344 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAkqCFmfPghOhAA1u10705aCcG60mm8TmLlVQ== 2024-04-18T23:10:21.344 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.8] 2024-04-18T23:10:21.344 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAoqCFmOv3BBRAA4sXqhcvUtlmiTj8hmrOcCg== 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.12] 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQArqCFmIMI6DBAApOxPbQuEnQbf87t2vp1M1Q== 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.1] 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAxqCFmjtc5ChAAa03o9TQtyIIK8Rv4KhLWew== 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.5] 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA0qCFmKMw3DhAA3HFi/yBJeJvdiQBH+4nxEg== 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.9] 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA3qCFmhj3PFBAA9hMOjI3PHVHsdRjO5vwyig== 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.13] 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQA6qCFmlZEpHBAAbYBdpEmkjmB00C9g0MuL5w== 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.2] 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBAqCFmFgWuKRAAOYjCWO62tr9uzI5DB/S95Q== 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.6] 2024-04-18T23:10:21.345 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBDqCFmhM8kMBAAAOGKKgm114/TEYmTNfoy1g== 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.10] 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBGqCFmGuowNxAAdGTESMJoBfJ/MhDOYAG1zw== 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.14] 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBKqCFmCxJkAhAAyvpJSdhleTwpkPRhKUl1Lg== 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:[mgr.x] 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAeqCFmrDyXMxAAzAZxP1AaDc06fT8t0rMebg== 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.3] 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBQqCFmYLq3BRAAJwYAF0c/kWce7BO51/IKzw== 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.7] 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBTqCFmCU5SChAAXRdjDM5S+9ZmdNONLkGczA== 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.11] 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBWqCFmueQOEhAAu+ktUlfQzEgzFqQXoRk5Ag== 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:[osd.15] 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQBZqCFm0tQwGBAAlKKhO9HVACxvK1RxBy9a8Q== 2024-04-18T23:10:21.346 INFO:teuthology.orchestra.run.smithi136.stdout:[client.0] 2024-04-18T23:10:21.347 INFO:teuthology.orchestra.run.smithi136.stdout: key = AQAeqCFm+x4ZORAA10dLPE537IDkfPfQ/2CBjQ== 2024-04-18T23:10:21.347 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-18T23:10:21.385 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-18T23:10:21.390 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-18T23:10:21.435 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.478 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.521 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.523 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.526 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.602 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.605 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.609 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.8 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.683 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.685 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.694 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.12 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.768 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.778 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.848 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.853 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.862 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.934 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:21.946 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.9 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.018 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.030 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.13 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.105 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.110 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.179 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.193 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.260 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.269 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.278 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.10 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.339 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.362 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.14 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.420 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-18T23:10:22.437 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-18T23:10:22.442 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-18T23:10:22.502 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.521 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.526 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.609 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.612 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.668 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.693 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.698 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.11 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.748 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.750 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.782 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.15 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-18T23:10:22.827 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-18T23:10:22.833 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-18T23:10:22.865 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-18T23:10:22.911 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-18T23:10:22.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-18T23:10:22.926 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-18T23:10:23.011 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-18T23:10:23.061 DEBUG:teuthology.orchestra.run.smithi120:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-18T23:10:23.075 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-18T23:10:23.168 DEBUG:teuthology.orchestra.run.smithi120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-18T23:10:23.178 DEBUG:teuthology.orchestra.run.smithi136:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-18T23:10:23.193 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-18T23:10:23.290 DEBUG:teuthology.orchestra.run.smithi136:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-18T23:10:23.301 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:10:23.303 DEBUG:teuthology.orchestra.run.smithi120:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:10:23.305 DEBUG:teuthology.orchestra.run.smithi136:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-18T23:10:23.349 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-18T23:10:23.349 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-18T23:10:23.350 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-18T23:10:23.352 INFO:tasks.ceph.mon.a:Started 2024-04-18T23:10:23.352 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-18T23:10:23.352 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-18T23:10:23.354 INFO:tasks.ceph.mon.b:Started 2024-04-18T23:10:23.355 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-18T23:10:23.355 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-18T23:10:23.395 INFO:tasks.ceph.mon.c:Started 2024-04-18T23:10:23.395 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-18T23:10:23.395 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-18T23:10:23.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-18T23:10:23.398 INFO:tasks.ceph.mgr.y:Started 2024-04-18T23:10:23.398 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-18T23:10:23.398 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-18T23:10:23.400 INFO:tasks.ceph.mgr.x:Started 2024-04-18T23:10:23.401 INFO:tasks.ceph:Setting crush tunables to default 2024-04-18T23:10:23.401 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd crush tunables default 2024-04-18T23:10:28.668 INFO:teuthology.orchestra.run.smithi073.stderr:adjusted tunables profile to default 2024-04-18T23:10:28.686 INFO:tasks.ceph:check_enable_crimson: False 2024-04-18T23:10:28.686 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-18T23:10:28.687 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:10:28.687 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-18T23:10:28.699 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:10:28.700 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-18T23:10:28.755 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:10:28.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-8/fsid of=/dev/stdout 2024-04-18T23:10:28.803 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:28.799+0000 7efd268fd180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-18T23:10:28.809 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2024-04-18T23:10:28.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-12/fsid of=/dev/stdout 2024-04-18T23:10:28.862 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:10:28.862 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-18T23:10:28.877 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:10:28.877 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-18T23:10:28.917 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:28.919+0000 7f6f7d414180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-18T23:10:28.931 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:10:28.931 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-9/fsid of=/dev/stdout 2024-04-18T23:10:28.984 DEBUG:teuthology.orchestra.run.smithi120:> set -ex 2024-04-18T23:10:28.984 DEBUG:teuthology.orchestra.run.smithi120:> sudo dd if=/var/lib/ceph/osd/ceph-13/fsid of=/dev/stdout 2024-04-18T23:10:29.040 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:10:29.040 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-18T23:10:29.054 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:10:29.054 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-18T23:10:29.110 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:10:29.110 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-10/fsid of=/dev/stdout 2024-04-18T23:10:29.164 DEBUG:teuthology.orchestra.run.smithi136:> set -ex 2024-04-18T23:10:29.165 DEBUG:teuthology.orchestra.run.smithi136:> sudo dd if=/var/lib/ceph/osd/ceph-14/fsid of=/dev/stdout 2024-04-18T23:10:29.220 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:29.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-18T23:10:29.235 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:29.235 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-18T23:10:29.289 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:29.289 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-11/fsid of=/dev/stdout 2024-04-18T23:10:29.308 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:29.307+0000 7efd268fd180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-18T23:10:29.343 DEBUG:teuthology.orchestra.run.smithi142:> set -ex 2024-04-18T23:10:29.343 DEBUG:teuthology.orchestra.run.smithi142:> sudo dd if=/var/lib/ceph/osd/ceph-15/fsid of=/dev/stdout 2024-04-18T23:10:29.396 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 3af51a22-0d99-447f-8e09-ef2a2077dcba 0 2024-04-18T23:10:29.434 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:29.435+0000 7f6f7d414180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-18T23:10:29.506 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:29.503+0000 7efd268fd180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-18T23:10:29.641 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:29.643+0000 7f6f7d414180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-18T23:10:29.652 INFO:teuthology.orchestra.run.smithi142.stdout:0 2024-04-18T23:10:29.671 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new f50deb0a-c98a-45ec-9ee3-be20aff17eb1 1 2024-04-18T23:10:29.887 INFO:teuthology.orchestra.run.smithi142.stdout:1 2024-04-18T23:10:29.905 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new b6556b6f-9ef8-462d-ae9f-b59713dec207 2 2024-04-18T23:10:30.056 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:30.055+0000 7efd268fd180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-18T23:10:30.105 INFO:teuthology.orchestra.run.smithi142.stdout:2 2024-04-18T23:10:30.110 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:30.107+0000 7efd268fd180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-18T23:10:30.123 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 2448b4d0-af77-4951-bcf3-f2752e7d13b5 3 2024-04-18T23:10:30.163 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:30.159+0000 7efd268fd180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-18T23:10:30.227 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:30.227+0000 7f6f7d414180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-18T23:10:30.243 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:30.235+0000 7efd268fd180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-18T23:10:30.282 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:30.283+0000 7f6f7d414180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-18T23:10:30.291 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:30.287+0000 7efd268fd180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-18T23:10:30.333 INFO:teuthology.orchestra.run.smithi142.stdout:3 2024-04-18T23:10:30.334 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:30.335+0000 7f6f7d414180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-18T23:10:30.350 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new d8ae2413-5474-40ca-b256-04b012595463 4 2024-04-18T23:10:30.368 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:30.367+0000 7efd268fd180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-18T23:10:30.414 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:30.415+0000 7f6f7d414180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-18T23:10:30.420 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:30.419+0000 7efd268fd180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-18T23:10:30.467 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:30.467+0000 7f6f7d414180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-18T23:10:30.543 INFO:teuthology.orchestra.run.smithi142.stdout:4 2024-04-18T23:10:30.544 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:30.547+0000 7f6f7d414180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-18T23:10:30.560 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 3515f9b0-470c-4db3-b456-a3985a36bf1f 5 2024-04-18T23:10:30.571 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:30.567+0000 7efd268fd180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-18T23:10:30.599 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:30.599+0000 7f6f7d414180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-18T23:10:30.633 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:30.631+0000 7efd268fd180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-18T23:10:30.751 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:30.751+0000 7f6f7d414180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-18T23:10:30.764 INFO:teuthology.orchestra.run.smithi142.stdout:5 2024-04-18T23:10:30.781 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 12d2f5e8-9eaa-4c21-a250-1db8c4e641cc 6 2024-04-18T23:10:30.814 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:30.815+0000 7f6f7d414180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-18T23:10:31.013 INFO:teuthology.orchestra.run.smithi142.stdout:6 2024-04-18T23:10:31.029 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new c9e5e983-106f-4d01-a153-cdf7d656a091 7 2024-04-18T23:10:31.267 INFO:teuthology.orchestra.run.smithi142.stdout:7 2024-04-18T23:10:31.284 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new a711d0fa-f5a7-4146-b280-8ee969afbb0f 8 2024-04-18T23:10:31.300 INFO:tasks.ceph.mgr.y.smithi073.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-18T23:10:31.300 INFO:tasks.ceph.mgr.y.smithi073.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-18T23:10:31.301 INFO:tasks.ceph.mgr.y.smithi073.stderr: from numpy import show_config as show_numpy_config 2024-04-18T23:10:31.363 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:31.359+0000 7efd268fd180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-18T23:10:31.447 INFO:tasks.ceph.mgr.x.smithi142.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2024-04-18T23:10:31.447 INFO:tasks.ceph.mgr.x.smithi142.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2024-04-18T23:10:31.447 INFO:tasks.ceph.mgr.x.smithi142.stderr: from numpy import show_config as show_numpy_config 2024-04-18T23:10:31.474 INFO:teuthology.orchestra.run.smithi142.stdout:8 2024-04-18T23:10:31.490 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 914af14e-f90d-42de-918a-d925f50ccfb2 9 2024-04-18T23:10:31.511 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:31.511+0000 7f6f7d414180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-18T23:10:31.538 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:31.535+0000 7efd268fd180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-18T23:10:31.679 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:31.679+0000 7f6f7d414180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-18T23:10:31.679 INFO:teuthology.orchestra.run.smithi142.stdout:9 2024-04-18T23:10:31.696 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 6ad314a3-6c86-46fd-b1a6-d20911da6d24 10 2024-04-18T23:10:31.701 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:31.699+0000 7efd268fd180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-18T23:10:31.764 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:31.763+0000 7efd268fd180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-18T23:10:31.849 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:31.851+0000 7f6f7d414180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-18T23:10:31.897 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:31.895+0000 7efd268fd180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-18T23:10:31.917 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:31.919+0000 7f6f7d414180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-18T23:10:31.926 INFO:teuthology.orchestra.run.smithi142.stdout:10 2024-04-18T23:10:31.942 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 2ea8b70d-e122-4c29-8742-18d341fd8668 11 2024-04-18T23:10:32.065 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:32.067+0000 7f6f7d414180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-18T23:10:32.147 INFO:teuthology.orchestra.run.smithi142.stdout:11 2024-04-18T23:10:32.165 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 24086ac0-a92d-400b-9b3d-f48992dc8679 12 2024-04-18T23:10:32.368 INFO:teuthology.orchestra.run.smithi142.stdout:12 2024-04-18T23:10:32.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new da98eea5-e373-4325-adda-75f747ad6a99 13 2024-04-18T23:10:32.524 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:32.523+0000 7efd268fd180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-18T23:10:32.581 INFO:teuthology.orchestra.run.smithi142.stdout:13 2024-04-18T23:10:32.598 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new d03df2d4-8d1c-4c2a-9719-d64ad780acc7 14 2024-04-18T23:10:32.601 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:32.599+0000 7efd268fd180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-18T23:10:32.721 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:32.723+0000 7f6f7d414180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-18T23:10:32.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:32.779+0000 7efd268fd180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-18T23:10:32.791 INFO:teuthology.orchestra.run.smithi142.stdout:14 2024-04-18T23:10:32.797 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:32.799+0000 7f6f7d414180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-18T23:10:32.807 DEBUG:teuthology.orchestra.run.smithi142:> sudo ceph --cluster ceph osd new 7aa92378-d336-41be-85f1-2eb582f54947 15 2024-04-18T23:10:32.857 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:32.855+0000 7efd268fd180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-18T23:10:32.983 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:32.983+0000 7f6f7d414180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-18T23:10:33.000 INFO:teuthology.orchestra.run.smithi142.stdout:15 2024-04-18T23:10:33.017 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-18T23:10:33.017 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-18T23:10:33.020 INFO:tasks.ceph.osd.0:Started 2024-04-18T23:10:33.020 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-18T23:10:33.020 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-18T23:10:33.022 INFO:tasks.ceph.osd.4:Started 2024-04-18T23:10:33.023 INFO:tasks.ceph.osd.8:Restarting daemon 2024-04-18T23:10:33.023 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 8 2024-04-18T23:10:33.025 INFO:tasks.ceph.osd.8:Started 2024-04-18T23:10:33.025 INFO:tasks.ceph.osd.12:Restarting daemon 2024-04-18T23:10:33.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 12 2024-04-18T23:10:33.027 INFO:tasks.ceph.osd.12:Started 2024-04-18T23:10:33.027 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-18T23:10:33.028 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-18T23:10:33.030 INFO:tasks.ceph.osd.1:Started 2024-04-18T23:10:33.030 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-18T23:10:33.031 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-18T23:10:33.033 INFO:tasks.ceph.osd.5:Started 2024-04-18T23:10:33.033 INFO:tasks.ceph.osd.9:Restarting daemon 2024-04-18T23:10:33.033 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 9 2024-04-18T23:10:33.035 INFO:tasks.ceph.osd.9:Started 2024-04-18T23:10:33.036 INFO:tasks.ceph.osd.13:Restarting daemon 2024-04-18T23:10:33.036 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 13 2024-04-18T23:10:33.038 INFO:tasks.ceph.osd.13:Started 2024-04-18T23:10:33.038 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-18T23:10:33.038 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-18T23:10:33.041 INFO:tasks.ceph.osd.2:Started 2024-04-18T23:10:33.041 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-18T23:10:33.041 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-18T23:10:33.044 INFO:tasks.ceph.osd.6:Started 2024-04-18T23:10:33.044 INFO:tasks.ceph.osd.10:Restarting daemon 2024-04-18T23:10:33.044 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 10 2024-04-18T23:10:33.046 INFO:tasks.ceph.osd.10:Started 2024-04-18T23:10:33.046 INFO:tasks.ceph.osd.14:Restarting daemon 2024-04-18T23:10:33.047 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 14 2024-04-18T23:10:33.049 INFO:tasks.ceph.osd.14:Started 2024-04-18T23:10:33.049 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-18T23:10:33.049 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-18T23:10:33.051 INFO:tasks.ceph.osd.3:Started 2024-04-18T23:10:33.051 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-18T23:10:33.051 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-18T23:10:33.053 INFO:tasks.ceph.osd.7:Started 2024-04-18T23:10:33.053 INFO:tasks.ceph.osd.11:Restarting daemon 2024-04-18T23:10:33.053 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 11 2024-04-18T23:10:33.055 INFO:tasks.ceph.osd.11:Started 2024-04-18T23:10:33.055 INFO:tasks.ceph.osd.15:Restarting daemon 2024-04-18T23:10:33.055 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-04-18T23:10:33.057 INFO:tasks.ceph.osd.15:Started 2024-04-18T23:10:33.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T23:10:33.059 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:33.059+0000 7f6f7d414180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-18T23:10:33.170 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:33.167+0000 7efd268fd180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-18T23:10:33.234 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:33.231+0000 7efd268fd180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-18T23:10:33.236 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:10:33.236 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":18,"fsid":"b8d0ffd9-6668-442b-a039-a24cea734f3e","created":"2024-04-18T23:10:28.548637+0000","modified":"2024-04-18T23:10:33.002063+0000","last_up_change":"0.000000","last_in_change":"2024-04-18T23:10:33.002063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3af51a22-0d99-447f-8e09-ef2a2077dcba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f50deb0a-c98a-45ec-9ee3-be20aff17eb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b6556b6f-9ef8-462d-ae9f-b59713dec207","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2448b4d0-af77-4951-bcf3-f2752e7d13b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d8ae2413-5474-40ca-b256-04b012595463","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3515f9b0-470c-4db3-b456-a3985a36bf1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"12d2f5e8-9eaa-4c21-a250-1db8c4e641cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c9e5e983-106f-4d01-a153-cdf7d656a091","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"a711d0fa-f5a7-4146-b280-8ee969afbb0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"914af14e-f90d-42de-918a-d925f50ccfb2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"6ad314a3-6c86-46fd-b1a6-d20911da6d24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"2ea8b70d-e122-4c29-8742-18d341fd8668","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"24086ac0-a92d-400b-9b3d-f48992dc8679","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"da98eea5-e373-4325-adda-75f747ad6a99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"d03df2d4-8d1c-4c2a-9719-d64ad780acc7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"7aa92378-d336-41be-85f1-2eb582f54947","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T23:10:33.252 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-18T23:10:33.252 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-18T23:10:33.363 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:33.363+0000 7f6f7d414180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-18T23:10:33.424 INFO:tasks.ceph.mgr.x.smithi142.stderr:2024-04-18T23:10:33.427+0000 7f6f7d414180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-18T23:10:33.657 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:10:33.655+0000 7fd599f177c0 -1 Falling back to public interface 2024-04-18T23:10:33.658 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:33.655+0000 7f9eb601d7c0 -1 Falling back to public interface 2024-04-18T23:10:33.659 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:33.655+0000 7f61622457c0 -1 Falling back to public interface 2024-04-18T23:10:33.659 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:10:33.655+0000 7ff3aa1a77c0 -1 Falling back to public interface 2024-04-18T23:10:33.660 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:33.659+0000 7f6161205640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6814/0: (98) Address already in use 2024-04-18T23:10:33.666 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:33.667+0000 7f2e5664f7c0 -1 Falling back to public interface 2024-04-18T23:10:33.666 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:33.667+0000 7f9f81e137c0 -1 Falling back to public interface 2024-04-18T23:10:33.666 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:33.667+0000 7f9f80dd3640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-18T23:10:33.667 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:33.667+0000 7fdb797dd7c0 -1 Falling back to public interface 2024-04-18T23:10:33.667 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:33.667+0000 7fdb7879d640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6805/0: (98) Address already in use 2024-04-18T23:10:33.675 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:33.675+0000 7f2fb124e7c0 -1 Falling back to public interface 2024-04-18T23:10:33.677 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:10:33.679+0000 7f60533437c0 -1 Falling back to public interface 2024-04-18T23:10:33.684 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:10:33.687+0000 7fc0760127c0 -1 Falling back to public interface 2024-04-18T23:10:33.704 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:33.707+0000 7fb5b495f7c0 -1 Falling back to public interface 2024-04-18T23:10:33.709 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:10:33.711+0000 7f26591a87c0 -1 Falling back to public interface 2024-04-18T23:10:33.740 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:10:33.741+0000 7f54bfdaa7c0 -1 Falling back to public interface 2024-04-18T23:10:33.740 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:10:33.741+0000 7f660c6f37c0 -1 Falling back to public interface 2024-04-18T23:10:33.741 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:33.741+0000 7f32a83787c0 -1 Falling back to public interface 2024-04-18T23:10:33.741 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:33.741+0000 7f14a3c277c0 -1 Falling back to public interface 2024-04-18T23:10:34.278 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:34.275+0000 7efccde49640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-18T23:10:34.279 INFO:tasks.ceph.mgr.y.smithi073.stderr:2024-04-18T23:10:34.275+0000 7efccde49640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-18T23:10:34.865 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-18T23:10:35.161 INFO:teuthology.misc.health.smithi073.stdout: 2024-04-18T23:10:35.161 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":18,"fsid":"b8d0ffd9-6668-442b-a039-a24cea734f3e","created":"2024-04-18T23:10:28.548637+0000","modified":"2024-04-18T23:10:33.002063+0000","last_up_change":"0.000000","last_in_change":"2024-04-18T23:10:33.002063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"3af51a22-0d99-447f-8e09-ef2a2077dcba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f50deb0a-c98a-45ec-9ee3-be20aff17eb1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b6556b6f-9ef8-462d-ae9f-b59713dec207","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2448b4d0-af77-4951-bcf3-f2752e7d13b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"d8ae2413-5474-40ca-b256-04b012595463","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"3515f9b0-470c-4db3-b456-a3985a36bf1f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"12d2f5e8-9eaa-4c21-a250-1db8c4e641cc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"c9e5e983-106f-4d01-a153-cdf7d656a091","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":8,"uuid":"a711d0fa-f5a7-4146-b280-8ee969afbb0f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":9,"uuid":"914af14e-f90d-42de-918a-d925f50ccfb2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":10,"uuid":"6ad314a3-6c86-46fd-b1a6-d20911da6d24","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":11,"uuid":"2ea8b70d-e122-4c29-8742-18d341fd8668","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":12,"uuid":"24086ac0-a92d-400b-9b3d-f48992dc8679","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":13,"uuid":"da98eea5-e373-4325-adda-75f747ad6a99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":14,"uuid":"d03df2d4-8d1c-4c2a-9719-d64ad780acc7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":15,"uuid":"7aa92378-d336-41be-85f1-2eb582f54947","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T23:10:35.178 DEBUG:teuthology.misc:0 of 16 OSDs are up 2024-04-18T23:10:37.412 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:37.411+0000 7f2e5664f7c0 -1 osd.5 0 log_to_monitors true 2024-04-18T23:10:37.414 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:37.414+0000 7f9f81e137c0 -1 osd.7 0 log_to_monitors true 2024-04-18T23:10:37.437 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:10:37.434+0000 7fd599f177c0 -1 osd.0 0 log_to_monitors true 2024-04-18T23:10:37.438 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:37.438+0000 7fdb797dd7c0 -1 osd.15 0 log_to_monitors true 2024-04-18T23:10:37.534 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:10:37.534+0000 7fc0760127c0 -1 osd.3 0 log_to_monitors true 2024-04-18T23:10:37.541 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:10:37.543+0000 7f60533437c0 -1 osd.9 0 log_to_monitors true 2024-04-18T23:10:37.550 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:10:37.546+0000 7ff3aa1a77c0 -1 osd.12 0 log_to_monitors true 2024-04-18T23:10:37.587 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:37.582+0000 7f9eb601d7c0 -1 osd.4 0 log_to_monitors true 2024-04-18T23:10:37.590 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:37.586+0000 7f61622457c0 -1 osd.8 0 log_to_monitors true 2024-04-18T23:10:37.591 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:37.590+0000 7fb5b495f7c0 -1 osd.11 0 log_to_monitors true 2024-04-18T23:10:37.591 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:10:37.592+0000 7f54bfdaa7c0 -1 osd.6 0 log_to_monitors true 2024-04-18T23:10:37.618 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:37.620+0000 7f32a83787c0 -1 osd.2 0 log_to_monitors true 2024-04-18T23:10:37.632 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:37.631+0000 7f2fb124e7c0 -1 osd.1 0 log_to_monitors true 2024-04-18T23:10:37.664 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:10:37.667+0000 7f26591a87c0 -1 osd.13 0 log_to_monitors true 2024-04-18T23:10:37.722 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:10:37.724+0000 7f660c6f37c0 -1 osd.10 0 log_to_monitors true 2024-04-18T23:10:37.742 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:37.744+0000 7f14a3c277c0 -1 osd.14 0 log_to_monitors true 2024-04-18T23:10:39.307 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:10:39.306+0000 7fc071798640 -1 osd.3 0 waiting for initial osdmap 2024-04-18T23:10:39.308 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:39.310+0000 7fb5b191b640 -1 osd.11 0 waiting for initial osdmap 2024-04-18T23:10:39.308 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:39.307+0000 7f2e5360b640 -1 osd.5 0 waiting for initial osdmap 2024-04-18T23:10:39.309 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:39.307+0000 7f2fac9d4640 -1 osd.1 0 waiting for initial osdmap 2024-04-18T23:10:39.311 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:10:39.306+0000 7fd59569d640 -1 osd.0 0 waiting for initial osdmap 2024-04-18T23:10:39.313 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:39.310+0000 7f615d9cb640 -1 osd.8 0 waiting for initial osdmap 2024-04-18T23:10:39.313 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:39.310+0000 7f9eb17a3640 -1 osd.4 0 waiting for initial osdmap 2024-04-18T23:10:39.321 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:39.322+0000 7f2fa9023640 -1 osd.1 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:39.326 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:39.322+0000 7f615a01a640 -1 osd.8 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:39.329 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:39.330+0000 7f2e4e424640 -1 osd.5 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:39.337 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:39.334+0000 7f9eaddf2640 -1 osd.4 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:39.351 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:10:39.350+0000 7fc06dde7640 -1 osd.3 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:39.351 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:39.350+0000 7fb5ac734640 -1 osd.11 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:39.390 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:10:39.386+0000 7fd591cec640 -1 osd.0 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:40.135 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:10:40.130+0000 7ff3a592d640 -1 osd.12 0 waiting for initial osdmap 2024-04-18T23:10:40.157 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:10:40.154+0000 7ff39df74640 -1 osd.12 20 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:40.506 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:40.506+0000 7fdb76799640 -1 osd.15 0 waiting for initial osdmap 2024-04-18T23:10:40.529 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:40.530+0000 7fdb6d5aa640 -1 osd.15 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:40.593 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:10:40.594+0000 7f604eac9640 -1 osd.9 0 waiting for initial osdmap 2024-04-18T23:10:40.601 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:10:40.602+0000 7f265492e640 -1 osd.13 0 waiting for initial osdmap 2024-04-18T23:10:40.621 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:40.622+0000 7f9f7d599640 -1 osd.7 0 waiting for initial osdmap 2024-04-18T23:10:40.637 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:10:40.638+0000 7f264cf75640 -1 osd.13 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:40.646 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:40.646+0000 7f9f75be0640 -1 osd.7 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:40.722 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:10:40.722+0000 7f6047110640 -1 osd.9 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:40.737 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:10:40.740+0000 7f66096af640 -1 osd.10 0 waiting for initial osdmap 2024-04-18T23:10:40.774 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:10:40.776+0000 7f66004c0640 -1 osd.10 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:40.789 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:10:40.788+0000 7f54bb530640 -1 osd.6 0 waiting for initial osdmap 2024-04-18T23:10:40.845 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:40.848+0000 7f14a0be3640 -1 osd.14 0 waiting for initial osdmap 2024-04-18T23:10:40.854 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:40.856+0000 7f32a3afe640 -1 osd.2 0 waiting for initial osdmap 2024-04-18T23:10:40.870 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:40.872+0000 7f329c145640 -1 osd.2 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:40.914 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:10:40.916+0000 7f54b3b77640 -1 osd.6 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:40.966 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:40.968+0000 7f14979f4640 -1 osd.14 21 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:10:42.788 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-18T23:10:43.088 INFO:teuthology.misc.health.smithi073.stdout: 2024-04-18T23:10:43.088 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":23,"fsid":"b8d0ffd9-6668-442b-a039-a24cea734f3e","created":"2024-04-18T23:10:28.548637+0000","modified":"2024-04-18T23:10:42.364658+0000","last_up_change":"2024-04-18T23:10:41.351544+0000","last_in_change":"2024-04-18T23:10:33.002063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T23:10:41.349753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3af51a22-0d99-447f-8e09-ef2a2077dcba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6803","nonce":169883533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6805","nonce":169883533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6817","nonce":169883533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6808","nonce":169883533}]},"public_addr":"172.21.15.73:6803/169883533","cluster_addr":"172.21.15.73:6805/169883533","heartbeat_back_addr":"172.21.15.73:6817/169883533","heartbeat_front_addr":"172.21.15.73:6808/169883533","state":["exists","up"]},{"osd":1,"uuid":"f50deb0a-c98a-45ec-9ee3-be20aff17eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6809","nonce":3147085605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6811","nonce":3147085605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6816","nonce":3147085605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6813","nonce":3147085605}]},"public_addr":"172.21.15.120:6809/3147085605","cluster_addr":"172.21.15.120:6811/3147085605","heartbeat_back_addr":"172.21.15.120:6816/3147085605","heartbeat_front_addr":"172.21.15.120:6813/3147085605","state":["exists","up"]},{"osd":2,"uuid":"b6556b6f-9ef8-462d-ae9f-b59713dec207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6809","nonce":1984461220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6819","nonce":1984461220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6831","nonce":1984461220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6823","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6827","nonce":1984461220}]},"public_addr":"172.21.15.136:6809/1984461220","cluster_addr":"172.21.15.136:6819/1984461220","heartbeat_back_addr":"172.21.15.136:6831/1984461220","heartbeat_front_addr":"172.21.15.136:6827/1984461220","state":["exists","up"]},{"osd":3,"uuid":"2448b4d0-af77-4951-bcf3-f2752e7d13b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6817","nonce":2108797520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6819","nonce":2108797520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6823","nonce":2108797520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6821","nonce":2108797520}]},"public_addr":"172.21.15.142:6817/2108797520","cluster_addr":"172.21.15.142:6819/2108797520","heartbeat_back_addr":"172.21.15.142:6823/2108797520","heartbeat_front_addr":"172.21.15.142:6821/2108797520","state":["exists","up"]},{"osd":4,"uuid":"d8ae2413-5474-40ca-b256-04b012595463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6812","nonce":1768990367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6820","nonce":1768990367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6831","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6833","nonce":1768990367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6823","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6827","nonce":1768990367}]},"public_addr":"172.21.15.73:6812/1768990367","cluster_addr":"172.21.15.73:6820/1768990367","heartbeat_back_addr":"172.21.15.73:6833/1768990367","heartbeat_front_addr":"172.21.15.73:6827/1768990367","state":["exists","up"]},{"osd":5,"uuid":"3515f9b0-470c-4db3-b456-a3985a36bf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6801","nonce":4018604021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6803","nonce":4018604021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6807","nonce":4018604021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6805","nonce":4018604021}]},"public_addr":"172.21.15.120:6801/4018604021","cluster_addr":"172.21.15.120:6803/4018604021","heartbeat_back_addr":"172.21.15.120:6807/4018604021","heartbeat_front_addr":"172.21.15.120:6805/4018604021","state":["exists","up"]},{"osd":6,"uuid":"12d2f5e8-9eaa-4c21-a250-1db8c4e641cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6803","nonce":2261503449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6811","nonce":2261503449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6826","nonce":2261503449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6818","nonce":2261503449}]},"public_addr":"172.21.15.136:6803/2261503449","cluster_addr":"172.21.15.136:6811/2261503449","heartbeat_back_addr":"172.21.15.136:6826/2261503449","heartbeat_front_addr":"172.21.15.136:6818/2261503449","state":["exists","up"]},{"osd":7,"uuid":"c9e5e983-106f-4d01-a153-cdf7d656a091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6804","nonce":1398734507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6807","nonce":1398734507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6814","nonce":1398734507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6810","nonce":1398734507}]},"public_addr":"172.21.15.142:6804/1398734507","cluster_addr":"172.21.15.142:6807/1398734507","heartbeat_back_addr":"172.21.15.142:6814/1398734507","heartbeat_front_addr":"172.21.15.142:6810/1398734507","state":["exists","up"]},{"osd":8,"uuid":"a711d0fa-f5a7-4146-b280-8ee969afbb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6815","nonce":786826444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6819","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6821","nonce":786826444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6829","nonce":786826444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6826","nonce":786826444}]},"public_addr":"172.21.15.73:6815/786826444","cluster_addr":"172.21.15.73:6821/786826444","heartbeat_back_addr":"172.21.15.73:6829/786826444","heartbeat_front_addr":"172.21.15.73:6826/786826444","state":["exists","up"]},{"osd":9,"uuid":"914af14e-f90d-42de-918a-d925f50ccfb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6817","nonce":2287493993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6819","nonce":2287493993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6823","nonce":2287493993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6821","nonce":2287493993}]},"public_addr":"172.21.15.120:6817/2287493993","cluster_addr":"172.21.15.120:6819/2287493993","heartbeat_back_addr":"172.21.15.120:6823/2287493993","heartbeat_front_addr":"172.21.15.120:6821/2287493993","state":["exists","up"]},{"osd":10,"uuid":"6ad314a3-6c86-46fd-b1a6-d20911da6d24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6802","nonce":1419708480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6808","nonce":1419708480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6825","nonce":1419708480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6817","nonce":1419708480}]},"public_addr":"172.21.15.136:6802/1419708480","cluster_addr":"172.21.15.136:6808/1419708480","heartbeat_back_addr":"172.21.15.136:6825/1419708480","heartbeat_front_addr":"172.21.15.136:6817/1419708480","state":["exists","up"]},{"osd":11,"uuid":"2ea8b70d-e122-4c29-8742-18d341fd8668","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6825","nonce":2871475141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6827","nonce":2871475141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6831","nonce":2871475141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6829","nonce":2871475141}]},"public_addr":"172.21.15.142:6825/2871475141","cluster_addr":"172.21.15.142:6827/2871475141","heartbeat_back_addr":"172.21.15.142:6831/2871475141","heartbeat_front_addr":"172.21.15.142:6829/2871475141","state":["exists","up"]},{"osd":12,"uuid":"24086ac0-a92d-400b-9b3d-f48992dc8679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6810","nonce":3885787537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6818","nonce":3885787537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6832","nonce":3885787537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6825","nonce":3885787537}]},"public_addr":"172.21.15.73:6810/3885787537","cluster_addr":"172.21.15.73:6818/3885787537","heartbeat_back_addr":"172.21.15.73:6832/3885787537","heartbeat_front_addr":"172.21.15.73:6825/3885787537","state":["exists","up"]},{"osd":13,"uuid":"da98eea5-e373-4325-adda-75f747ad6a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6825","nonce":2634549424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6827","nonce":2634549424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6831","nonce":2634549424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6829","nonce":2634549424}]},"public_addr":"172.21.15.120:6825/2634549424","cluster_addr":"172.21.15.120:6827/2634549424","heartbeat_back_addr":"172.21.15.120:6831/2634549424","heartbeat_front_addr":"172.21.15.120:6829/2634549424","state":["exists","up"]},{"osd":14,"uuid":"d03df2d4-8d1c-4c2a-9719-d64ad780acc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6810","nonce":381469963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6816","nonce":381469963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6829","nonce":381469963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6824","nonce":381469963}]},"public_addr":"172.21.15.136:6810/381469963","cluster_addr":"172.21.15.136:6816/381469963","heartbeat_back_addr":"172.21.15.136:6829/381469963","heartbeat_front_addr":"172.21.15.136:6824/381469963","state":["exists","up"]},{"osd":15,"uuid":"7aa92378-d336-41be-85f1-2eb582f54947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6801","nonce":458186762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6806","nonce":458186762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6815","nonce":458186762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6811","nonce":458186762}]},"public_addr":"172.21.15.142:6801/458186762","cluster_addr":"172.21.15.142:6806/458186762","heartbeat_back_addr":"172.21.15.142:6815/458186762","heartbeat_front_addr":"172.21.15.142:6811/458186762","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.485101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.664021+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.600492+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.498355+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.557595+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.388760+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.627022+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.384945+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.585718+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.569751+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.751111+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.621068+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.521391+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.679869+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.725910+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.440535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T23:10:43.107 DEBUG:teuthology.misc:16 of 16 OSDs are up 2024-04-18T23:10:43.107 INFO:tasks.ceph:Creating RBD pool 2024-04-18T23:10:43.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-18T23:10:44.386 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'rbd' created 2024-04-18T23:10:44.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-18T23:10:45.388 INFO:teuthology.orchestra.run.smithi073.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-18T23:10:45.415 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-18T23:10:45.415 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-18T23:10:45.415 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-18T23:10:45.416 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-18T23:10:45.416 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-18T23:10:45.760 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:10:45.778 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":4,"flags":0,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":2940797671},{"type":"v1","addr":"172.21.15.73:6801","nonce":2940797671}]},"active_addr":"172.21.15.73:6801/2940797671","active_change":"2024-04-18T23:10:33.258985+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4110,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":2838686474}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":2109741114}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":4267559990}]}]} 2024-04-18T23:10:45.782 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-18T23:10:45.782 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-18T23:10:45.782 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T23:10:46.076 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:10:46.076 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"fsid":"b8d0ffd9-6668-442b-a039-a24cea734f3e","created":"2024-04-18T23:10:28.548637+0000","modified":"2024-04-18T23:10:45.384008+0000","last_up_change":"2024-04-18T23:10:41.351544+0000","last_in_change":"2024-04-18T23:10:33.002063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T23:10:41.349753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T23:10:43.462157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3af51a22-0d99-447f-8e09-ef2a2077dcba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6803","nonce":169883533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6805","nonce":169883533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6817","nonce":169883533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6808","nonce":169883533}]},"public_addr":"172.21.15.73:6803/169883533","cluster_addr":"172.21.15.73:6805/169883533","heartbeat_back_addr":"172.21.15.73:6817/169883533","heartbeat_front_addr":"172.21.15.73:6808/169883533","state":["exists","up"]},{"osd":1,"uuid":"f50deb0a-c98a-45ec-9ee3-be20aff17eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6809","nonce":3147085605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6811","nonce":3147085605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6816","nonce":3147085605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6813","nonce":3147085605}]},"public_addr":"172.21.15.120:6809/3147085605","cluster_addr":"172.21.15.120:6811/3147085605","heartbeat_back_addr":"172.21.15.120:6816/3147085605","heartbeat_front_addr":"172.21.15.120:6813/3147085605","state":["exists","up"]},{"osd":2,"uuid":"b6556b6f-9ef8-462d-ae9f-b59713dec207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6809","nonce":1984461220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6819","nonce":1984461220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6831","nonce":1984461220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6823","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6827","nonce":1984461220}]},"public_addr":"172.21.15.136:6809/1984461220","cluster_addr":"172.21.15.136:6819/1984461220","heartbeat_back_addr":"172.21.15.136:6831/1984461220","heartbeat_front_addr":"172.21.15.136:6827/1984461220","state":["exists","up"]},{"osd":3,"uuid":"2448b4d0-af77-4951-bcf3-f2752e7d13b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6817","nonce":2108797520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6819","nonce":2108797520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6823","nonce":2108797520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6821","nonce":2108797520}]},"public_addr":"172.21.15.142:6817/2108797520","cluster_addr":"172.21.15.142:6819/2108797520","heartbeat_back_addr":"172.21.15.142:6823/2108797520","heartbeat_front_addr":"172.21.15.142:6821/2108797520","state":["exists","up"]},{"osd":4,"uuid":"d8ae2413-5474-40ca-b256-04b012595463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6812","nonce":1768990367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6820","nonce":1768990367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6831","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6833","nonce":1768990367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6823","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6827","nonce":1768990367}]},"public_addr":"172.21.15.73:6812/1768990367","cluster_addr":"172.21.15.73:6820/1768990367","heartbeat_back_addr":"172.21.15.73:6833/1768990367","heartbeat_front_addr":"172.21.15.73:6827/1768990367","state":["exists","up"]},{"osd":5,"uuid":"3515f9b0-470c-4db3-b456-a3985a36bf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6801","nonce":4018604021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6803","nonce":4018604021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6807","nonce":4018604021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6805","nonce":4018604021}]},"public_addr":"172.21.15.120:6801/4018604021","cluster_addr":"172.21.15.120:6803/4018604021","heartbeat_back_addr":"172.21.15.120:6807/4018604021","heartbeat_front_addr":"172.21.15.120:6805/4018604021","state":["exists","up"]},{"osd":6,"uuid":"12d2f5e8-9eaa-4c21-a250-1db8c4e641cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6803","nonce":2261503449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6811","nonce":2261503449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6826","nonce":2261503449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6818","nonce":2261503449}]},"public_addr":"172.21.15.136:6803/2261503449","cluster_addr":"172.21.15.136:6811/2261503449","heartbeat_back_addr":"172.21.15.136:6826/2261503449","heartbeat_front_addr":"172.21.15.136:6818/2261503449","state":["exists","up"]},{"osd":7,"uuid":"c9e5e983-106f-4d01-a153-cdf7d656a091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6804","nonce":1398734507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6807","nonce":1398734507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6814","nonce":1398734507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6810","nonce":1398734507}]},"public_addr":"172.21.15.142:6804/1398734507","cluster_addr":"172.21.15.142:6807/1398734507","heartbeat_back_addr":"172.21.15.142:6814/1398734507","heartbeat_front_addr":"172.21.15.142:6810/1398734507","state":["exists","up"]},{"osd":8,"uuid":"a711d0fa-f5a7-4146-b280-8ee969afbb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6815","nonce":786826444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6819","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6821","nonce":786826444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6829","nonce":786826444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6826","nonce":786826444}]},"public_addr":"172.21.15.73:6815/786826444","cluster_addr":"172.21.15.73:6821/786826444","heartbeat_back_addr":"172.21.15.73:6829/786826444","heartbeat_front_addr":"172.21.15.73:6826/786826444","state":["exists","up"]},{"osd":9,"uuid":"914af14e-f90d-42de-918a-d925f50ccfb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6817","nonce":2287493993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6819","nonce":2287493993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6823","nonce":2287493993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6821","nonce":2287493993}]},"public_addr":"172.21.15.120:6817/2287493993","cluster_addr":"172.21.15.120:6819/2287493993","heartbeat_back_addr":"172.21.15.120:6823/2287493993","heartbeat_front_addr":"172.21.15.120:6821/2287493993","state":["exists","up"]},{"osd":10,"uuid":"6ad314a3-6c86-46fd-b1a6-d20911da6d24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6802","nonce":1419708480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6808","nonce":1419708480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6825","nonce":1419708480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6817","nonce":1419708480}]},"public_addr":"172.21.15.136:6802/1419708480","cluster_addr":"172.21.15.136:6808/1419708480","heartbeat_back_addr":"172.21.15.136:6825/1419708480","heartbeat_front_addr":"172.21.15.136:6817/1419708480","state":["exists","up"]},{"osd":11,"uuid":"2ea8b70d-e122-4c29-8742-18d341fd8668","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6825","nonce":2871475141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6827","nonce":2871475141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6831","nonce":2871475141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6829","nonce":2871475141}]},"public_addr":"172.21.15.142:6825/2871475141","cluster_addr":"172.21.15.142:6827/2871475141","heartbeat_back_addr":"172.21.15.142:6831/2871475141","heartbeat_front_addr":"172.21.15.142:6829/2871475141","state":["exists","up"]},{"osd":12,"uuid":"24086ac0-a92d-400b-9b3d-f48992dc8679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6810","nonce":3885787537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6818","nonce":3885787537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6832","nonce":3885787537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6825","nonce":3885787537}]},"public_addr":"172.21.15.73:6810/3885787537","cluster_addr":"172.21.15.73:6818/3885787537","heartbeat_back_addr":"172.21.15.73:6832/3885787537","heartbeat_front_addr":"172.21.15.73:6825/3885787537","state":["exists","up"]},{"osd":13,"uuid":"da98eea5-e373-4325-adda-75f747ad6a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6825","nonce":2634549424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6827","nonce":2634549424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6831","nonce":2634549424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6829","nonce":2634549424}]},"public_addr":"172.21.15.120:6825/2634549424","cluster_addr":"172.21.15.120:6827/2634549424","heartbeat_back_addr":"172.21.15.120:6831/2634549424","heartbeat_front_addr":"172.21.15.120:6829/2634549424","state":["exists","up"]},{"osd":14,"uuid":"d03df2d4-8d1c-4c2a-9719-d64ad780acc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6810","nonce":381469963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6816","nonce":381469963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6829","nonce":381469963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6824","nonce":381469963}]},"public_addr":"172.21.15.136:6810/381469963","cluster_addr":"172.21.15.136:6816/381469963","heartbeat_back_addr":"172.21.15.136:6829/381469963","heartbeat_front_addr":"172.21.15.136:6824/381469963","state":["exists","up"]},{"osd":15,"uuid":"7aa92378-d336-41be-85f1-2eb582f54947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6801","nonce":458186762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6806","nonce":458186762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6815","nonce":458186762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6811","nonce":458186762}]},"public_addr":"172.21.15.142:6801/458186762","cluster_addr":"172.21.15.142:6806/458186762","heartbeat_back_addr":"172.21.15.142:6815/458186762","heartbeat_front_addr":"172.21.15.142:6811/458186762","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.485101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.664021+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.600492+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.498355+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.557595+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.388760+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.627022+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.384945+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.585718+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.569751+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.751111+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.621068+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.521391+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.679869+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.725910+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.440535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T23:10:46.095 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-18T23:10:46.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T23:10:46.402 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:10:46.402 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"fsid":"b8d0ffd9-6668-442b-a039-a24cea734f3e","created":"2024-04-18T23:10:28.548637+0000","modified":"2024-04-18T23:10:45.384008+0000","last_up_change":"2024-04-18T23:10:41.351544+0000","last_in_change":"2024-04-18T23:10:33.002063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T23:10:41.349753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T23:10:43.462157+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"26","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3af51a22-0d99-447f-8e09-ef2a2077dcba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6803","nonce":169883533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6805","nonce":169883533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6817","nonce":169883533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6808","nonce":169883533}]},"public_addr":"172.21.15.73:6803/169883533","cluster_addr":"172.21.15.73:6805/169883533","heartbeat_back_addr":"172.21.15.73:6817/169883533","heartbeat_front_addr":"172.21.15.73:6808/169883533","state":["exists","up"]},{"osd":1,"uuid":"f50deb0a-c98a-45ec-9ee3-be20aff17eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6809","nonce":3147085605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6811","nonce":3147085605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6816","nonce":3147085605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6813","nonce":3147085605}]},"public_addr":"172.21.15.120:6809/3147085605","cluster_addr":"172.21.15.120:6811/3147085605","heartbeat_back_addr":"172.21.15.120:6816/3147085605","heartbeat_front_addr":"172.21.15.120:6813/3147085605","state":["exists","up"]},{"osd":2,"uuid":"b6556b6f-9ef8-462d-ae9f-b59713dec207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6809","nonce":1984461220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6819","nonce":1984461220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6831","nonce":1984461220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6823","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6827","nonce":1984461220}]},"public_addr":"172.21.15.136:6809/1984461220","cluster_addr":"172.21.15.136:6819/1984461220","heartbeat_back_addr":"172.21.15.136:6831/1984461220","heartbeat_front_addr":"172.21.15.136:6827/1984461220","state":["exists","up"]},{"osd":3,"uuid":"2448b4d0-af77-4951-bcf3-f2752e7d13b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6817","nonce":2108797520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6819","nonce":2108797520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6823","nonce":2108797520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6821","nonce":2108797520}]},"public_addr":"172.21.15.142:6817/2108797520","cluster_addr":"172.21.15.142:6819/2108797520","heartbeat_back_addr":"172.21.15.142:6823/2108797520","heartbeat_front_addr":"172.21.15.142:6821/2108797520","state":["exists","up"]},{"osd":4,"uuid":"d8ae2413-5474-40ca-b256-04b012595463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6812","nonce":1768990367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6820","nonce":1768990367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6831","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6833","nonce":1768990367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6823","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6827","nonce":1768990367}]},"public_addr":"172.21.15.73:6812/1768990367","cluster_addr":"172.21.15.73:6820/1768990367","heartbeat_back_addr":"172.21.15.73:6833/1768990367","heartbeat_front_addr":"172.21.15.73:6827/1768990367","state":["exists","up"]},{"osd":5,"uuid":"3515f9b0-470c-4db3-b456-a3985a36bf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6801","nonce":4018604021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6803","nonce":4018604021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6807","nonce":4018604021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6805","nonce":4018604021}]},"public_addr":"172.21.15.120:6801/4018604021","cluster_addr":"172.21.15.120:6803/4018604021","heartbeat_back_addr":"172.21.15.120:6807/4018604021","heartbeat_front_addr":"172.21.15.120:6805/4018604021","state":["exists","up"]},{"osd":6,"uuid":"12d2f5e8-9eaa-4c21-a250-1db8c4e641cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6803","nonce":2261503449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6811","nonce":2261503449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6826","nonce":2261503449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6818","nonce":2261503449}]},"public_addr":"172.21.15.136:6803/2261503449","cluster_addr":"172.21.15.136:6811/2261503449","heartbeat_back_addr":"172.21.15.136:6826/2261503449","heartbeat_front_addr":"172.21.15.136:6818/2261503449","state":["exists","up"]},{"osd":7,"uuid":"c9e5e983-106f-4d01-a153-cdf7d656a091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6804","nonce":1398734507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6807","nonce":1398734507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6814","nonce":1398734507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6810","nonce":1398734507}]},"public_addr":"172.21.15.142:6804/1398734507","cluster_addr":"172.21.15.142:6807/1398734507","heartbeat_back_addr":"172.21.15.142:6814/1398734507","heartbeat_front_addr":"172.21.15.142:6810/1398734507","state":["exists","up"]},{"osd":8,"uuid":"a711d0fa-f5a7-4146-b280-8ee969afbb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6815","nonce":786826444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6819","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6821","nonce":786826444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6829","nonce":786826444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6826","nonce":786826444}]},"public_addr":"172.21.15.73:6815/786826444","cluster_addr":"172.21.15.73:6821/786826444","heartbeat_back_addr":"172.21.15.73:6829/786826444","heartbeat_front_addr":"172.21.15.73:6826/786826444","state":["exists","up"]},{"osd":9,"uuid":"914af14e-f90d-42de-918a-d925f50ccfb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6817","nonce":2287493993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6819","nonce":2287493993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6823","nonce":2287493993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6821","nonce":2287493993}]},"public_addr":"172.21.15.120:6817/2287493993","cluster_addr":"172.21.15.120:6819/2287493993","heartbeat_back_addr":"172.21.15.120:6823/2287493993","heartbeat_front_addr":"172.21.15.120:6821/2287493993","state":["exists","up"]},{"osd":10,"uuid":"6ad314a3-6c86-46fd-b1a6-d20911da6d24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6802","nonce":1419708480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6808","nonce":1419708480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6825","nonce":1419708480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6817","nonce":1419708480}]},"public_addr":"172.21.15.136:6802/1419708480","cluster_addr":"172.21.15.136:6808/1419708480","heartbeat_back_addr":"172.21.15.136:6825/1419708480","heartbeat_front_addr":"172.21.15.136:6817/1419708480","state":["exists","up"]},{"osd":11,"uuid":"2ea8b70d-e122-4c29-8742-18d341fd8668","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6825","nonce":2871475141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6827","nonce":2871475141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6831","nonce":2871475141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6829","nonce":2871475141}]},"public_addr":"172.21.15.142:6825/2871475141","cluster_addr":"172.21.15.142:6827/2871475141","heartbeat_back_addr":"172.21.15.142:6831/2871475141","heartbeat_front_addr":"172.21.15.142:6829/2871475141","state":["exists","up"]},{"osd":12,"uuid":"24086ac0-a92d-400b-9b3d-f48992dc8679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6810","nonce":3885787537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6818","nonce":3885787537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6832","nonce":3885787537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6825","nonce":3885787537}]},"public_addr":"172.21.15.73:6810/3885787537","cluster_addr":"172.21.15.73:6818/3885787537","heartbeat_back_addr":"172.21.15.73:6832/3885787537","heartbeat_front_addr":"172.21.15.73:6825/3885787537","state":["exists","up"]},{"osd":13,"uuid":"da98eea5-e373-4325-adda-75f747ad6a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6825","nonce":2634549424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6827","nonce":2634549424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6831","nonce":2634549424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6829","nonce":2634549424}]},"public_addr":"172.21.15.120:6825/2634549424","cluster_addr":"172.21.15.120:6827/2634549424","heartbeat_back_addr":"172.21.15.120:6831/2634549424","heartbeat_front_addr":"172.21.15.120:6829/2634549424","state":["exists","up"]},{"osd":14,"uuid":"d03df2d4-8d1c-4c2a-9719-d64ad780acc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6810","nonce":381469963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6816","nonce":381469963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6829","nonce":381469963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6824","nonce":381469963}]},"public_addr":"172.21.15.136:6810/381469963","cluster_addr":"172.21.15.136:6816/381469963","heartbeat_back_addr":"172.21.15.136:6829/381469963","heartbeat_front_addr":"172.21.15.136:6824/381469963","state":["exists","up"]},{"osd":15,"uuid":"7aa92378-d336-41be-85f1-2eb582f54947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6801","nonce":458186762}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6806","nonce":458186762}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6815","nonce":458186762}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":458186762},{"type":"v1","addr":"172.21.15.142:6811","nonce":458186762}]},"public_addr":"172.21.15.142:6801/458186762","cluster_addr":"172.21.15.142:6806/458186762","heartbeat_back_addr":"172.21.15.142:6815/458186762","heartbeat_front_addr":"172.21.15.142:6811/458186762","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.485101+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.664021+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.600492+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.498355+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.557595+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.388760+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.627022+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.384945+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.585718+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.569751+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.751111+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.621068+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.521391+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.679869+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.725910+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:10:38.440535+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T23:10:46.420 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-18T23:10:46.421 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-18T23:10:46.421 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-18T23:10:46.422 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-18T23:10:46.422 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-18T23:10:46.423 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-18T23:10:46.423 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-18T23:10:46.423 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-18T23:10:46.424 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-04-18T23:10:46.424 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-04-18T23:10:46.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-04-18T23:10:46.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-04-18T23:10:46.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-04-18T23:10:46.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-04-18T23:10:46.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-04-18T23:10:46.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-04-18T23:10:46.810 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:46.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-18T23:10:46.865 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-04-18T23:10:46.865 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-18T23:10:46.882 INFO:teuthology.orchestra.run.smithi073.stdout:90194313220 2024-04-18T23:10:46.883 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-18T23:10:46.890 INFO:teuthology.orchestra.run.smithi073.stdout:90194313221 2024-04-18T23:10:46.890 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-18T23:10:46.906 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-04-18T23:10:46.906 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-18T23:10:46.954 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:46.955 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-18T23:10:46.964 INFO:teuthology.orchestra.run.smithi073.stdout:90194313220 2024-04-18T23:10:46.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-18T23:10:46.965 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:46.965 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-18T23:10:46.966 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-04-18T23:10:46.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-18T23:10:46.973 INFO:teuthology.orchestra.run.smithi073.stdout:90194313219 2024-04-18T23:10:46.973 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-18T23:10:46.985 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:46.986 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-18T23:10:46.990 INFO:teuthology.orchestra.run.smithi073.stdout:90194313221 2024-04-18T23:10:46.990 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-18T23:10:47.020 INFO:teuthology.orchestra.run.smithi073.stdout:90194313219 2024-04-18T23:10:47.020 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-18T23:10:47.024 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:47.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-18T23:10:47.027 INFO:teuthology.orchestra.run.smithi073.stdout:90194313221 2024-04-18T23:10:47.027 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-18T23:10:47.042 INFO:teuthology.orchestra.run.smithi073.stdout:90194313220 2024-04-18T23:10:47.042 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-18T23:10:47.786 INFO:teuthology.orchestra.run.smithi073.stdout:90194313220 2024-04-18T23:10:47.799 INFO:teuthology.orchestra.run.smithi073.stdout:90194313219 2024-04-18T23:10:47.844 INFO:teuthology.orchestra.run.smithi073.stdout:90194313221 2024-04-18T23:10:47.848 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.12 2024-04-18T23:10:47.848 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:47.852 INFO:teuthology.orchestra.run.smithi073.stdout:90194313221 2024-04-18T23:10:47.852 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.11 2024-04-18T23:10:47.853 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:47.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.4 2024-04-18T23:10:47.892 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:47.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.5 2024-04-18T23:10:47.903 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:47.915 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:47.950 INFO:teuthology.orchestra.run.smithi073.stdout:90194313220 2024-04-18T23:10:47.956 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.6 2024-04-18T23:10:47.957 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:47.963 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-04-18T23:10:47.990 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-04-18T23:10:47.997 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.1 2024-04-18T23:10:47.997 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.002 INFO:teuthology.orchestra.run.smithi073.stdout:90194313221 2024-04-18T23:10:48.017 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:48.017 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.15 2024-04-18T23:10:48.018 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.023 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.9 2024-04-18T23:10:48.023 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313221 got 90194313221 for osd.3 2024-04-18T23:10:48.045 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.045 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.2 2024-04-18T23:10:48.046 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.049 INFO:teuthology.orchestra.run.smithi073.stdout:90194313219 2024-04-18T23:10:48.054 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:48.055 INFO:teuthology.orchestra.run.smithi073.stdout:94489280516 2024-04-18T23:10:48.075 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.7 2024-04-18T23:10:48.075 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.081 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313219 got 90194313219 for osd.8 2024-04-18T23:10:48.081 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.085 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280516 got 94489280516 for osd.10 2024-04-18T23:10:48.085 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.087 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:48.094 INFO:teuthology.orchestra.run.smithi073.stdout:94489280517 2024-04-18T23:10:48.105 INFO:teuthology.orchestra.run.smithi073.stdout:90194313220 2024-04-18T23:10:48.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.13 2024-04-18T23:10:48.113 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.113 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280517 got 94489280517 for osd.14 2024-04-18T23:10:48.113 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.125 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313220 got 90194313220 for osd.0 2024-04-18T23:10:48.125 DEBUG:teuthology.parallel:result is None 2024-04-18T23:10:48.125 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-18T23:10:48.125 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:10:48.434 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:10:48.434 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:10:48.451 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-04-18T23:10:47.468283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":437880,"kb_used_data":5072,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511560,"statfs":{"total":1546188226560,"available":1545739837440,"internally_reserved":0,"allocated":5193728,"data_stored":2520704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25520,"internal_metadata":442473552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.062608"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.548884+0000","last_change":"2024-04-18T23:10:45.598826+0000","last_active":"2024-04-18T23:10:46.548884+0000","last_peered":"2024-04-18T23:10:46.548884+0000","last_clean":"2024-04-18T23:10:46.548884+0000","last_became_active":"2024-04-18T23:10:45.598172+0000","last_became_peered":"2024-04-18T23:10:45.598172+0000","last_unstale":"2024-04-18T23:10:46.548884+0000","last_undegraded":"2024-04-18T23:10:46.548884+0000","last_fullsized":"2024-04-18T23:10:46.548884+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.086797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.544911+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:10:46.544911+0000","last_peered":"2024-04-18T23:10:46.544911+0000","last_clean":"2024-04-18T23:10:46.544911+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:10:46.544911+0000","last_undegraded":"2024-04-18T23:10:46.544911+0000","last_fullsized":"2024-04-18T23:10:46.544911+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:50.312119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.549345+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:10:46.549345+0000","last_peered":"2024-04-18T23:10:46.549345+0000","last_clean":"2024-04-18T23:10:46.549345+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:10:46.549345+0000","last_undegraded":"2024-04-18T23:10:46.549345+0000","last_fullsized":"2024-04-18T23:10:46.549345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.454390+0000","last_change":"2024-04-18T23:10:46.454390+0000","last_active":"2024-04-18T23:10:46.454390+0000","last_peered":"2024-04-18T23:10:46.454390+0000","last_clean":"2024-04-18T23:10:46.454390+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:10:46.454390+0000","last_undegraded":"2024-04-18T23:10:46.454390+0000","last_fullsized":"2024-04-18T23:10:46.454390+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:04.782253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.648169+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:10:46.648169+0000","last_peered":"2024-04-18T23:10:46.648169+0000","last_clean":"2024-04-18T23:10:46.648169+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:10:46.648169+0000","last_undegraded":"2024-04-18T23:10:46.648169+0000","last_fullsized":"2024-04-18T23:10:46.648169+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:46.679756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.511459+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:10:46.511459+0000","last_peered":"2024-04-18T23:10:46.511459+0000","last_clean":"2024-04-18T23:10:46.511459+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:10:46.511459+0000","last_undegraded":"2024-04-18T23:10:46.511459+0000","last_fullsized":"2024-04-18T23:10:46.511459+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:02.503215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.648151+0000","last_change":"2024-04-18T23:10:45.497972+0000","last_active":"2024-04-18T23:10:46.648151+0000","last_peered":"2024-04-18T23:10:46.648151+0000","last_clean":"2024-04-18T23:10:46.648151+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:10:46.648151+0000","last_undegraded":"2024-04-18T23:10:46.648151+0000","last_fullsized":"2024-04-18T23:10:46.648151+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.381771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.549444+0000","last_change":"2024-04-18T23:10:45.601489+0000","last_active":"2024-04-18T23:10:46.549444+0000","last_peered":"2024-04-18T23:10:46.549444+0000","last_clean":"2024-04-18T23:10:46.549444+0000","last_became_active":"2024-04-18T23:10:45.600830+0000","last_became_peered":"2024-04-18T23:10:45.600830+0000","last_unstale":"2024-04-18T23:10:46.549444+0000","last_undegraded":"2024-04-18T23:10:46.549444+0000","last_fullsized":"2024-04-18T23:10:46.549444+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.491572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'63","reported_seq":105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.875866+0000","last_change":"2024-04-18T23:10:42.682223+0000","last_active":"2024-04-18T23:10:46.875866+0000","last_peered":"2024-04-18T23:10:46.875866+0000","last_clean":"2024-04-18T23:10:46.875866+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:10:46.875866+0000","last_undegraded":"2024-04-18T23:10:46.875866+0000","last_fullsized":"2024-04-18T23:10:46.875866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:10:41.351544+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:44.525791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":684032,"data_stored":510216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,6,8,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[2,4,5,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,3,4,5,6,8,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":684032,"data_stored":510216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[4,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:10:48.452 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:10:48.718 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:10:48.718 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:10:48.735 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2024-04-18T23:10:47.468283+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":437880,"kb_used_data":5072,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511560,"statfs":{"total":1546188226560,"available":1545739837440,"internally_reserved":0,"allocated":5193728,"data_stored":2520704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25520,"internal_metadata":442473552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.062608"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.548884+0000","last_change":"2024-04-18T23:10:45.598826+0000","last_active":"2024-04-18T23:10:46.548884+0000","last_peered":"2024-04-18T23:10:46.548884+0000","last_clean":"2024-04-18T23:10:46.548884+0000","last_became_active":"2024-04-18T23:10:45.598172+0000","last_became_peered":"2024-04-18T23:10:45.598172+0000","last_unstale":"2024-04-18T23:10:46.548884+0000","last_undegraded":"2024-04-18T23:10:46.548884+0000","last_fullsized":"2024-04-18T23:10:46.548884+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.086797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.544911+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:10:46.544911+0000","last_peered":"2024-04-18T23:10:46.544911+0000","last_clean":"2024-04-18T23:10:46.544911+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:10:46.544911+0000","last_undegraded":"2024-04-18T23:10:46.544911+0000","last_fullsized":"2024-04-18T23:10:46.544911+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:50.312119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.549345+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:10:46.549345+0000","last_peered":"2024-04-18T23:10:46.549345+0000","last_clean":"2024-04-18T23:10:46.549345+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:10:46.549345+0000","last_undegraded":"2024-04-18T23:10:46.549345+0000","last_fullsized":"2024-04-18T23:10:46.549345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.454390+0000","last_change":"2024-04-18T23:10:46.454390+0000","last_active":"2024-04-18T23:10:46.454390+0000","last_peered":"2024-04-18T23:10:46.454390+0000","last_clean":"2024-04-18T23:10:46.454390+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:10:46.454390+0000","last_undegraded":"2024-04-18T23:10:46.454390+0000","last_fullsized":"2024-04-18T23:10:46.454390+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:04.782253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.648169+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:10:46.648169+0000","last_peered":"2024-04-18T23:10:46.648169+0000","last_clean":"2024-04-18T23:10:46.648169+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:10:46.648169+0000","last_undegraded":"2024-04-18T23:10:46.648169+0000","last_fullsized":"2024-04-18T23:10:46.648169+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:46.679756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.511459+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:10:46.511459+0000","last_peered":"2024-04-18T23:10:46.511459+0000","last_clean":"2024-04-18T23:10:46.511459+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:10:46.511459+0000","last_undegraded":"2024-04-18T23:10:46.511459+0000","last_fullsized":"2024-04-18T23:10:46.511459+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:02.503215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.648151+0000","last_change":"2024-04-18T23:10:45.497972+0000","last_active":"2024-04-18T23:10:46.648151+0000","last_peered":"2024-04-18T23:10:46.648151+0000","last_clean":"2024-04-18T23:10:46.648151+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:10:46.648151+0000","last_undegraded":"2024-04-18T23:10:46.648151+0000","last_fullsized":"2024-04-18T23:10:46.648151+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.381771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.549444+0000","last_change":"2024-04-18T23:10:45.601489+0000","last_active":"2024-04-18T23:10:46.549444+0000","last_peered":"2024-04-18T23:10:46.549444+0000","last_clean":"2024-04-18T23:10:46.549444+0000","last_became_active":"2024-04-18T23:10:45.600830+0000","last_became_peered":"2024-04-18T23:10:45.600830+0000","last_unstale":"2024-04-18T23:10:46.549444+0000","last_undegraded":"2024-04-18T23:10:46.549444+0000","last_fullsized":"2024-04-18T23:10:46.549444+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.491572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'63","reported_seq":105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.875866+0000","last_change":"2024-04-18T23:10:42.682223+0000","last_active":"2024-04-18T23:10:46.875866+0000","last_peered":"2024-04-18T23:10:46.875866+0000","last_clean":"2024-04-18T23:10:46.875866+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:10:46.875866+0000","last_undegraded":"2024-04-18T23:10:46.875866+0000","last_fullsized":"2024-04-18T23:10:46.875866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:10:41.351544+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:44.525791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":684032,"data_stored":510216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,6,8,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[2,4,5,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,3,4,5,6,8,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":684032,"data_stored":510216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[4,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:10:48.735 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-18T23:10:48.736 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-18T23:10:48.736 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-18T23:10:48.736 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-18T23:10:49.042 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:10:49.042 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-18T23:10:49.059 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-18T23:10:49.060 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-18T23:10:49.070 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-18T23:10:49.070 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'global': {'mon election default strategy': 3, 'osd debug inject dispatch delay duration': 0.1, 'osd debug inject dispatch delay probability': 0.1, 'osd_async_recovery_min_cost': 1}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon scrub interval': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug reject backfill probability': 0.3, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max backfills': 3, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd scrub max interval': 120, 'osd scrub min interval': 60, 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'slow request', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED'], 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1a6a3e742fd81c6d70be70552499bff767652c32'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'd0bf63dd179c05751e4be49477389aa290c73089'}} 2024-04-18T23:10:49.070 INFO:tasks.thrashosds:config is {'aggressive_pg_num_changes': False, 'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'min_in': 8, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-18T23:10:49.070 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-18T23:10:49.070 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-18T23:10:49.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:10:49.342 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:10:49.343 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:10:49.359 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-04-18T23:10:49.265636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":437880,"kb_used_data":5072,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511560,"statfs":{"total":1546188226560,"available":1545739837440,"internally_reserved":0,"allocated":5193728,"data_stored":2520704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25520,"internal_metadata":442473552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.859961"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.548884+0000","last_change":"2024-04-18T23:10:45.598826+0000","last_active":"2024-04-18T23:10:46.548884+0000","last_peered":"2024-04-18T23:10:46.548884+0000","last_clean":"2024-04-18T23:10:46.548884+0000","last_became_active":"2024-04-18T23:10:45.598172+0000","last_became_peered":"2024-04-18T23:10:45.598172+0000","last_unstale":"2024-04-18T23:10:46.548884+0000","last_undegraded":"2024-04-18T23:10:46.548884+0000","last_fullsized":"2024-04-18T23:10:46.548884+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.086797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.544911+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:10:46.544911+0000","last_peered":"2024-04-18T23:10:46.544911+0000","last_clean":"2024-04-18T23:10:46.544911+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:10:46.544911+0000","last_undegraded":"2024-04-18T23:10:46.544911+0000","last_fullsized":"2024-04-18T23:10:46.544911+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:50.312119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.549345+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:10:46.549345+0000","last_peered":"2024-04-18T23:10:46.549345+0000","last_clean":"2024-04-18T23:10:46.549345+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:10:46.549345+0000","last_undegraded":"2024-04-18T23:10:46.549345+0000","last_fullsized":"2024-04-18T23:10:46.549345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.454390+0000","last_change":"2024-04-18T23:10:46.454390+0000","last_active":"2024-04-18T23:10:46.454390+0000","last_peered":"2024-04-18T23:10:46.454390+0000","last_clean":"2024-04-18T23:10:46.454390+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:10:46.454390+0000","last_undegraded":"2024-04-18T23:10:46.454390+0000","last_fullsized":"2024-04-18T23:10:46.454390+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:04.782253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.648169+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:10:46.648169+0000","last_peered":"2024-04-18T23:10:46.648169+0000","last_clean":"2024-04-18T23:10:46.648169+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:10:46.648169+0000","last_undegraded":"2024-04-18T23:10:46.648169+0000","last_fullsized":"2024-04-18T23:10:46.648169+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:46.679756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.511459+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:10:46.511459+0000","last_peered":"2024-04-18T23:10:46.511459+0000","last_clean":"2024-04-18T23:10:46.511459+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:10:46.511459+0000","last_undegraded":"2024-04-18T23:10:46.511459+0000","last_fullsized":"2024-04-18T23:10:46.511459+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:02.503215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.648151+0000","last_change":"2024-04-18T23:10:45.497972+0000","last_active":"2024-04-18T23:10:46.648151+0000","last_peered":"2024-04-18T23:10:46.648151+0000","last_clean":"2024-04-18T23:10:46.648151+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:10:46.648151+0000","last_undegraded":"2024-04-18T23:10:46.648151+0000","last_fullsized":"2024-04-18T23:10:46.648151+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.381771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.549444+0000","last_change":"2024-04-18T23:10:45.601489+0000","last_active":"2024-04-18T23:10:46.549444+0000","last_peered":"2024-04-18T23:10:46.549444+0000","last_clean":"2024-04-18T23:10:46.549444+0000","last_became_active":"2024-04-18T23:10:45.600830+0000","last_became_peered":"2024-04-18T23:10:45.600830+0000","last_unstale":"2024-04-18T23:10:46.549444+0000","last_undegraded":"2024-04-18T23:10:46.549444+0000","last_fullsized":"2024-04-18T23:10:46.549444+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.491572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'63","reported_seq":105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.875866+0000","last_change":"2024-04-18T23:10:42.682223+0000","last_active":"2024-04-18T23:10:46.875866+0000","last_peered":"2024-04-18T23:10:46.875866+0000","last_clean":"2024-04-18T23:10:46.875866+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:10:46.875866+0000","last_undegraded":"2024-04-18T23:10:46.875866+0000","last_fullsized":"2024-04-18T23:10:46.875866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:10:41.351544+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:44.525791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":684032,"data_stored":510216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,6,8,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[2,4,5,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,3,4,5,6,8,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":684032,"data_stored":510216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[4,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:10:49.360 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:10:49.638 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:10:49.638 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:10:49.654 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":17,"stamp":"2024-04-18T23:10:49.265636+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":13,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":437880,"kb_used_data":5072,"kb_used_omap":24,"kb_used_meta":432103,"kb_avail":1509511560,"statfs":{"total":1546188226560,"available":1545739837440,"internally_reserved":0,"allocated":5193728,"data_stored":2520704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":25520,"internal_metadata":442473552},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":9,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.859961"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.548884+0000","last_change":"2024-04-18T23:10:45.598826+0000","last_active":"2024-04-18T23:10:46.548884+0000","last_peered":"2024-04-18T23:10:46.548884+0000","last_clean":"2024-04-18T23:10:46.548884+0000","last_became_active":"2024-04-18T23:10:45.598172+0000","last_became_peered":"2024-04-18T23:10:45.598172+0000","last_unstale":"2024-04-18T23:10:46.548884+0000","last_undegraded":"2024-04-18T23:10:46.548884+0000","last_fullsized":"2024-04-18T23:10:46.548884+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.086797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.544911+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:10:46.544911+0000","last_peered":"2024-04-18T23:10:46.544911+0000","last_clean":"2024-04-18T23:10:46.544911+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:10:46.544911+0000","last_undegraded":"2024-04-18T23:10:46.544911+0000","last_fullsized":"2024-04-18T23:10:46.544911+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:50.312119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.549345+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:10:46.549345+0000","last_peered":"2024-04-18T23:10:46.549345+0000","last_clean":"2024-04-18T23:10:46.549345+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:10:46.549345+0000","last_undegraded":"2024-04-18T23:10:46.549345+0000","last_fullsized":"2024-04-18T23:10:46.549345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.585124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.454390+0000","last_change":"2024-04-18T23:10:46.454390+0000","last_active":"2024-04-18T23:10:46.454390+0000","last_peered":"2024-04-18T23:10:46.454390+0000","last_clean":"2024-04-18T23:10:46.454390+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:10:46.454390+0000","last_undegraded":"2024-04-18T23:10:46.454390+0000","last_fullsized":"2024-04-18T23:10:46.454390+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:04.782253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.648169+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:10:46.648169+0000","last_peered":"2024-04-18T23:10:46.648169+0000","last_clean":"2024-04-18T23:10:46.648169+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:10:46.648169+0000","last_undegraded":"2024-04-18T23:10:46.648169+0000","last_fullsized":"2024-04-18T23:10:46.648169+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:46.679756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.511459+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:10:46.511459+0000","last_peered":"2024-04-18T23:10:46.511459+0000","last_clean":"2024-04-18T23:10:46.511459+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:10:46.511459+0000","last_undegraded":"2024-04-18T23:10:46.511459+0000","last_fullsized":"2024-04-18T23:10:46.511459+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:02.503215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.648151+0000","last_change":"2024-04-18T23:10:45.497972+0000","last_active":"2024-04-18T23:10:46.648151+0000","last_peered":"2024-04-18T23:10:46.648151+0000","last_clean":"2024-04-18T23:10:46.648151+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:10:46.648151+0000","last_undegraded":"2024-04-18T23:10:46.648151+0000","last_fullsized":"2024-04-18T23:10:46.648151+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.381771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.549444+0000","last_change":"2024-04-18T23:10:45.601489+0000","last_active":"2024-04-18T23:10:46.549444+0000","last_peered":"2024-04-18T23:10:46.549444+0000","last_clean":"2024-04-18T23:10:46.549444+0000","last_became_active":"2024-04-18T23:10:45.600830+0000","last_became_peered":"2024-04-18T23:10:45.600830+0000","last_unstale":"2024-04-18T23:10:46.549444+0000","last_undegraded":"2024-04-18T23:10:46.549444+0000","last_fullsized":"2024-04-18T23:10:46.549444+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.491572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"27'63","reported_seq":105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-18T23:10:46.875866+0000","last_change":"2024-04-18T23:10:42.682223+0000","last_active":"2024-04-18T23:10:46.875866+0000","last_peered":"2024-04-18T23:10:46.875866+0000","last_clean":"2024-04-18T23:10:46.875866+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:10:46.875866+0000","last_undegraded":"2024-04-18T23:10:46.875866+0000","last_fullsized":"2024-04-18T23:10:46.875866+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:10:41.351544+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:44.525791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":459280,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":84,"num_read_kb":71,"num_write":97,"num_write_kb":1110,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":802816,"data_stored":795680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":63,"ondisk_log_size":63,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":22,"seq":94489280517,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,6,7,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27296,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344544,"statfs":{"total":96636764160,"available":96608813056,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,7,8,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":684032,"data_stored":510216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,1,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313220,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,11,12,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27284,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344556,"statfs":{"total":96636764160,"available":96608825344,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313221,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,2,4,5,6,8,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,8,11,12],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313221,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313219,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[2,4,5,6,7,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313219,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27312,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344528,"statfs":{"total":96636764160,"available":96608796672,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,3,4,5,6,8,10,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313220,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27276,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344564,"statfs":{"total":96636764160,"available":96608833536,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280517,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27880,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343960,"statfs":{"total":96636764160,"available":96608215040,"internally_reserved":0,"allocated":684032,"data_stored":510216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,3,4,5,6,7,8,9,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280517,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27308,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344532,"statfs":{"total":96636764160,"available":96608800768,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280518,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27288,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344552,"statfs":{"total":96636764160,"available":96608821248,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[4,6,8,9,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27280,"kb_used_data":260,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344560,"statfs":{"total":96636764160,"available":96608829440,"internally_reserved":0,"allocated":266240,"data_stored":103252,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,8,10,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280517,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27300,"kb_used_data":276,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344540,"statfs":{"total":96636764160,"available":96608808960,"internally_reserved":0,"allocated":282624,"data_stored":112376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,7,8,9,11,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":401408,"data_stored":397840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:10:49.655 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-18T23:10:49.655 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-18T23:10:49.928 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 28 2024-04-18T23:10:49.928 INFO:teuthology.orchestra.run.smithi073.stdout:fsid b8d0ffd9-6668-442b-a039-a24cea734f3e 2024-04-18T23:10:49.928 INFO:teuthology.orchestra.run.smithi073.stdout:created 2024-04-18T23:10:28.548637+0000 2024-04-18T23:10:49.928 INFO:teuthology.orchestra.run.smithi073.stdout:modified 2024-04-18T23:10:47.408200+0000 2024-04-18T23:10:49.928 INFO:teuthology.orchestra.run.smithi073.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-18T23:10:49.928 INFO:teuthology.orchestra.run.smithi073.stdout:crush_version 4 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:full_ratio 0.95 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:backfillfull_ratio 0.9 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:nearfull_ratio 0.85 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:require_min_compat_client luminous 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:min_compat_client jewel 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:require_osd_release squid 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:stretch_mode_enabled false 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 24 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 15.38 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 28 flags hashpspool stripe_width 0 application rbd read_balance_score 4.02 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:max_osd 16 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6802/169883533,v1:172.21.15.73:6803/169883533] [v2:172.21.15.73:6804/169883533,v1:172.21.15.73:6805/169883533] exists,up 3af51a22-0d99-447f-8e09-ef2a2077dcba 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/3147085605,v1:172.21.15.120:6809/3147085605] [v2:172.21.15.120:6810/3147085605,v1:172.21.15.120:6811/3147085605] exists,up f50deb0a-c98a-45ec-9ee3-be20aff17eb1 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6804/1984461220,v1:172.21.15.136:6809/1984461220] [v2:172.21.15.136:6815/1984461220,v1:172.21.15.136:6819/1984461220] exists,up b6556b6f-9ef8-462d-ae9f-b59713dec207 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/2108797520,v1:172.21.15.142:6817/2108797520] [v2:172.21.15.142:6818/2108797520,v1:172.21.15.142:6819/2108797520] exists,up 2448b4d0-af77-4951-bcf3-f2752e7d13b5 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6809/1768990367,v1:172.21.15.73:6812/1768990367] [v2:172.21.15.73:6816/1768990367,v1:172.21.15.73:6820/1768990367] exists,up d8ae2413-5474-40ca-b256-04b012595463 2024-04-18T23:10:49.929 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/4018604021,v1:172.21.15.120:6801/4018604021] [v2:172.21.15.120:6802/4018604021,v1:172.21.15.120:6803/4018604021] exists,up 3515f9b0-470c-4db3-b456-a3985a36bf1f 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6801/2261503449,v1:172.21.15.136:6803/2261503449] [v2:172.21.15.136:6807/2261503449,v1:172.21.15.136:6811/2261503449] exists,up 12d2f5e8-9eaa-4c21-a250-1db8c4e641cc 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6802/1398734507,v1:172.21.15.142:6804/1398734507] [v2:172.21.15.142:6805/1398734507,v1:172.21.15.142:6807/1398734507] exists,up c9e5e983-106f-4d01-a153-cdf7d656a091 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6811/786826444,v1:172.21.15.73:6815/786826444] [v2:172.21.15.73:6819/786826444,v1:172.21.15.73:6821/786826444] exists,up a711d0fa-f5a7-4146-b280-8ee969afbb0f 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6815/2287493993,v1:172.21.15.120:6817/2287493993] [v2:172.21.15.120:6818/2287493993,v1:172.21.15.120:6819/2287493993] exists,up 914af14e-f90d-42de-918a-d925f50ccfb2 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/1419708480,v1:172.21.15.136:6802/1419708480] [v2:172.21.15.136:6805/1419708480,v1:172.21.15.136:6808/1419708480] exists,up 6ad314a3-6c86-46fd-b1a6-d20911da6d24 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/2871475141,v1:172.21.15.142:6825/2871475141] [v2:172.21.15.142:6826/2871475141,v1:172.21.15.142:6827/2871475141] exists,up 2ea8b70d-e122-4c29-8742-18d341fd8668 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6807/3885787537,v1:172.21.15.73:6810/3885787537] [v2:172.21.15.73:6814/3885787537,v1:172.21.15.73:6818/3885787537] exists,up 24086ac0-a92d-400b-9b3d-f48992dc8679 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/2634549424,v1:172.21.15.120:6825/2634549424] [v2:172.21.15.120:6826/2634549424,v1:172.21.15.120:6827/2634549424] exists,up da98eea5-e373-4325-adda-75f747ad6a99 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6806/381469963,v1:172.21.15.136:6810/381469963] [v2:172.21.15.136:6813/381469963,v1:172.21.15.136:6816/381469963] exists,up d03df2d4-8d1c-4c2a-9719-d64ad780acc7 2024-04-18T23:10:49.930 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/458186762,v1:172.21.15.142:6801/458186762] [v2:172.21.15.142:6803/458186762,v1:172.21.15.142:6806/458186762] exists,up 7aa92378-d336-41be-85f1-2eb582f54947 2024-04-18T23:10:49.946 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6802/169883533,v1:172.21.15.73:6803/169883533] [v2:172.21.15.73:6804/169883533,v1:172.21.15.73:6805/169883533] exists,up 3af51a22-0d99-447f-8e09-ef2a2077dcba', 'osd.1 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6808/3147085605,v1:172.21.15.120:6809/3147085605] [v2:172.21.15.120:6810/3147085605,v1:172.21.15.120:6811/3147085605] exists,up f50deb0a-c98a-45ec-9ee3-be20aff17eb1', 'osd.2 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6804/1984461220,v1:172.21.15.136:6809/1984461220] [v2:172.21.15.136:6815/1984461220,v1:172.21.15.136:6819/1984461220] exists,up b6556b6f-9ef8-462d-ae9f-b59713dec207', 'osd.3 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6816/2108797520,v1:172.21.15.142:6817/2108797520] [v2:172.21.15.142:6818/2108797520,v1:172.21.15.142:6819/2108797520] exists,up 2448b4d0-af77-4951-bcf3-f2752e7d13b5', 'osd.4 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6809/1768990367,v1:172.21.15.73:6812/1768990367] [v2:172.21.15.73:6816/1768990367,v1:172.21.15.73:6820/1768990367] exists,up d8ae2413-5474-40ca-b256-04b012595463', 'osd.5 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6800/4018604021,v1:172.21.15.120:6801/4018604021] [v2:172.21.15.120:6802/4018604021,v1:172.21.15.120:6803/4018604021] exists,up 3515f9b0-470c-4db3-b456-a3985a36bf1f', 'osd.6 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6801/2261503449,v1:172.21.15.136:6803/2261503449] [v2:172.21.15.136:6807/2261503449,v1:172.21.15.136:6811/2261503449] exists,up 12d2f5e8-9eaa-4c21-a250-1db8c4e641cc', 'osd.7 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6802/1398734507,v1:172.21.15.142:6804/1398734507] [v2:172.21.15.142:6805/1398734507,v1:172.21.15.142:6807/1398734507] exists,up c9e5e983-106f-4d01-a153-cdf7d656a091', 'osd.8 up in weight 1 up_from 21 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6811/786826444,v1:172.21.15.73:6815/786826444] [v2:172.21.15.73:6819/786826444,v1:172.21.15.73:6821/786826444] exists,up a711d0fa-f5a7-4146-b280-8ee969afbb0f', 'osd.9 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6815/2287493993,v1:172.21.15.120:6817/2287493993] [v2:172.21.15.120:6818/2287493993,v1:172.21.15.120:6819/2287493993] exists,up 914af14e-f90d-42de-918a-d925f50ccfb2', 'osd.10 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6800/1419708480,v1:172.21.15.136:6802/1419708480] [v2:172.21.15.136:6805/1419708480,v1:172.21.15.136:6808/1419708480] exists,up 6ad314a3-6c86-46fd-b1a6-d20911da6d24', 'osd.11 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6824/2871475141,v1:172.21.15.142:6825/2871475141] [v2:172.21.15.142:6826/2871475141,v1:172.21.15.142:6827/2871475141] exists,up 2ea8b70d-e122-4c29-8742-18d341fd8668', 'osd.12 up in weight 1 up_from 21 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.73:6807/3885787537,v1:172.21.15.73:6810/3885787537] [v2:172.21.15.73:6814/3885787537,v1:172.21.15.73:6818/3885787537] exists,up 24086ac0-a92d-400b-9b3d-f48992dc8679', 'osd.13 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.120:6824/2634549424,v1:172.21.15.120:6825/2634549424] [v2:172.21.15.120:6826/2634549424,v1:172.21.15.120:6827/2634549424] exists,up da98eea5-e373-4325-adda-75f747ad6a99', 'osd.14 up in weight 1 up_from 22 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.136:6806/381469963,v1:172.21.15.136:6810/381469963] [v2:172.21.15.136:6813/381469963,v1:172.21.15.136:6816/381469963] exists,up d03df2d4-8d1c-4c2a-9719-d64ad780acc7', 'osd.15 up in weight 1 up_from 22 up_thru 25 down_at 0 last_clean_interval [0,0) [v2:172.21.15.142:6800/458186762,v1:172.21.15.142:6801/458186762] [v2:172.21.15.142:6803/458186762,v1:172.21.15.142:6806/458186762] exists,up 7aa92378-d336-41be-85f1-2eb582f54947'] 2024-04-18T23:10:49.947 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-18T23:10:50.056 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:10:50.056 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "mon.a", 2024-04-18T23:10:50.056 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster": "ceph", 2024-04-18T23:10:50.056 INFO:teuthology.orchestra.run.smithi073.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "admin_socket_mode": "", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_client_required": "cephx, none", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_cluster_required": "cephx", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_debug": "true", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_service_required": "cephx", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_supported": "cephx", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio": "true", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio_poll_ms": "250", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_aio_reap_max": "16", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_async_discard": "false", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_block_size": "4096", 2024-04-18T23:10:50.057 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_aio": "false", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_debug_inflight_ios": "false", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_enable_discard": "false", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_flock_retry": "3", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_inject_crash": "0", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_ioring": "false", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_ioring_hipri": "false", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bdev_type": "", 2024-04-18T23:10:50.058 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_alloc_size": "1048576", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_allocator": "hybrid", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_buffered_io": "true", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_check_for_zeros": "false", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_compact_log_sync": "false", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_max_log_runway": "4194304", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_max_prefetch": "1048576", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_min_flush_size": "524288", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_min_log_runway": "1048576", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_replay_recovery": "false", 2024-04-18T23:10:50.059 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluefs_sync_write": "false", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_allocation_from_file": "true", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_allocator": "hybrid", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_create": "true", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_db_create": "false", 2024-04-18T23:10:50.060 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_db_path": "", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_db_size": "0", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_path": "", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_preallocate_file": "false", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_size": "107374182400", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_wal_create": "false", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_wal_path": "", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_block_wal_size": "100663296", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs": "true", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-18T23:10:50.061 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_autotune": "true", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_size": "0", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cache_type": "2q", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_clone_cow": "true", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-18T23:10:50.062 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_mode": "none", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_csum_type": "crc32c", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_freelist": "false", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-18T23:10:50.063 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_small_allocations": "0", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_default_buffered_read": "true", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_default_buffered_write": "false", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-18T23:10:50.064 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_elastic_shared_blobs": "true", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fail_eio": "false", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mount": "false", 2024-04-18T23:10:50.065 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_umount": "false", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_ignore_data_csum": "false", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-18T23:10:50.066 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_log_op_age": "5.000000", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_alloc_size": "0", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_blob_size": "0", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_max_deferred_txc": "32", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_min_alloc_size": "0", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_nid_prealloc": "1024", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_retry_disk_reads": "3", 2024-04-18T23:10:50.067 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_cf": "true", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_spdk_mem": "512", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-18T23:10:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-18T23:10:50.069 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-18T23:10:50.069 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-18T23:10:50.069 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-18T23:10:50.069 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-18T23:10:50.069 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_tracing": "false", 2024-04-18T23:10:50.069 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-18T23:10:50.069 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "bluestore_zero_block_detection": "false", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-18T23:10:50.070 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_cluster_require_signatures": "false", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_cluster_require_version": "2", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_require_signatures": "false", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_require_version": "2", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_service_require_signatures": "false", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_service_require_version": "2", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "cephx_sign_messages": "true", 2024-04-18T23:10:50.071 INFO:teuthology.orchestra.run.smithi073.stdout: "chdir": "", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_acl_type": "", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_asio_thread_count": "2", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_cache_mid": "0.750000", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_cache_size": "16384", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_caps_release_delay": "5", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_check_pool_perm": "true", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_force_sync_read": "false", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_getattr_caps": "false", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_debug_inject_tick_delay": "0", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_die_on_failed_remount": "false", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_dirsize_rbytes": "true", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_force_lazyio": "false", 2024-04-18T23:10:50.072 INFO:teuthology.orchestra.run.smithi073.stdout: "client_fs": "", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_inject_release_failure": "false", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_max_inline_size": "4096", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mds_namespace": "", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_metadata": "", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mount_gid": "-1", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mount_timeout": "300", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mount_uid": "-1", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_mountpoint": "/", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_notify_timeout": "10", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc": "true", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_max_dirty": "104857600", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_max_objects": "1000", 2024-04-18T23:10:50.073 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_size": "209715200", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_oc_target_dirty": "8388608", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_permissions": "true", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_quota": "true", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_quota_df": "true", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_readahead_max_bytes": "0", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_readahead_max_periods": "4", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_readahead_min": "131072", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_reconnect_stale": "false", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_shutdown_timeout": "30", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_snapdir": ".snap", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_tick_interval": "1", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_trace": "", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_try_dentry_invalidate": "false", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_use_faked_inos": "false", 2024-04-18T23:10:50.074 INFO:teuthology.orchestra.run.smithi073.stdout: "client_use_random_mds": "false", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_graylog": "false", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_graylog_port": "12201", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_monitors": "default=true", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_syslog": "false", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "clog_to_syslog_level": "info", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_addr": "-", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_network": "", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_network_interface": "", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zlib_isal": "false", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zlib_level": "5", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zlib_winsize": "-15", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "compressor_zstd_level": "1", 2024-04-18T23:10:50.075 INFO:teuthology.orchestra.run.smithi073.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_alien_op_num_threads": "6", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_osd_obc_lru_size": "512", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "crimson_seastar_cpu_cores": "", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_location": "", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_location_hook": "", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "crush_location_hook_timeout": "10", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "daemonize": "false", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "daos_pool": "tank", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "dbstore_db_name_prefix": "dbstore", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_alienstore": "0/5", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_allow_any_pool_priority": "false", 2024-04-18T23:10:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_asok": "1/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_asok_assert_abort": "false", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_asserts_on_shutdown": "true", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_auth": "20/20", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_bdev": "1/3", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_bluefs": "1/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_bluestore": "1/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_buffer": "0/1", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_ceph_exporter": "1/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_cephfs_mirror": "0/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_cephsqlite": "0/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_client": "0/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_compressor": "1/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_context": "0/1", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_crimson_interrupt": "0/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_crush": "1/1", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_crypto": "1/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_cyanstore": "0/5", 2024-04-18T23:10:50.077 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_deliberately_leak_memory": "false", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_disable_randomized_ping": "false", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_dpdk": "1/5", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_eventtrace": "1/5", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_filer": "0/1", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_filestore": "1/3", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_finisher": "1/1", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_fuse": "1/5", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_heartbeat_testing_span": "0", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_heartbeatmap": "1/5", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_javaclient": "1/5", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_journal": "1/3", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_journaler": "0/5", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_kstore": "1/5", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_lockdep": "0/1", 2024-04-18T23:10:50.078 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mclock": "1/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds": "1/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_balancer": "1/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_locker": "1/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_log": "1/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_log_expire": "1/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_migrator": "1/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mds_quiesce": "3/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_memstore": "1/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mgr": "2/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mgrc": "1/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_mon": "20/20", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_monc": "0/10", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_ms": "1/1", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_none": "0/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_objclass": "0/5", 2024-04-18T23:10:50.079 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_objectcacher": "0/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_objecter": "0/1", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_optracker": "0/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_osd": "1/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_paxos": "20/20", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_perfcounter": "1/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_prioritycache": "1/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rados": "0/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd": "0/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd_mirror": "0/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd_pwl": "0/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rbd_replay": "0/5", 2024-04-18T23:10:50.080 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_refs": "0/0", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_reserver": "1/1", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw": "1/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_access": "1/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_datacache": "1/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_dbstore": "1/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_flight": "1/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_lifecycle": "1/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rgw_sync": "1/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_rocksdb": "4/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore": "0/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_backref": "0/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_cache": "0/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_cleaner": "0/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_device": "0/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_epm": "0/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-04-18T23:10:50.081 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_journal": "0/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_lba": "0/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_odata": "0/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_omap": "0/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_onode": "0/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_t": "0/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_seastore_tm": "0/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_striper": "0/1", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_test": "0/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_throttle": "1/1", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_timer": "0/1", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_tp": "0/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "debug_trace": "1/5", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "device_failure_prediction_mode": "none", 2024-04-18T23:10:50.082 INFO:teuthology.orchestra.run.smithi073.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "erasure_code_dir": "/usr/lib/ceph/erasure-code", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_graylog": "false", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_journald": "false", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_stderr": "true", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "err_to_syslog": "false", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event_tracing": "false", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_addr": "0.0.0.0", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_http_port": "9926", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_prio_limit": "5", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_sort_metrics": "true", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "exporter_stats_period": "5", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "fake_statfs_for_testing": "0", 2024-04-18T23:10:50.083 INFO:teuthology.orchestra.run.smithi073.stdout: "fatal_signal_handlers": "true", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filer_max_purge_ops": "10", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filer_max_truncate_ops": "128", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_apply_finisher_threads": "1", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_blackhole": "false", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_btrfs_clone_range": "true", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_btrfs_snap": "true", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_caller_concurrency": "10", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_collect_device_partition_information": "true", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_commit_timeout": "600.000000", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_inject_read_err": "false", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_omap_check": "false", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_random_read_err": "0.000000", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_debug_verify_split": "false", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_dump_file": "", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-18T23:10:50.084 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fadvise": "true", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fail_eio": "true", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fd_cache_shards": "16", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fd_cache_size": "128", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fiemap": "false", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fiemap_threshold": "4096", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_inject_stall": "0", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_journal_parallel": "false", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_journal_trailing": "false", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_journal_writeahead": "false", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_kill_at": "0", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-18T23:10:50.085 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs": "0", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size": "0", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_merge_threshold": "-10", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-18T23:10:50.086 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_odsync_write": "false", 2024-04-18T23:10:50.087 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_omap_backend": "rocksdb", 2024-04-18T23:10:50.087 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_omap_backend_path": "", 2024-04-18T23:10:50.087 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-18T23:10:50.087 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-18T23:10:50.087 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-18T23:10:50.087 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_op_thread_timeout": "60", 2024-04-18T23:10:50.087 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_op_threads": "2", 2024-04-18T23:10:50.087 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_punch_hole": "false", 2024-04-18T23:10:50.087 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_queue_max_ops": "50", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_seek_data_hole": "false", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_sloppy_crc": "false", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_splice": "false", 2024-04-18T23:10:50.088 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_split_multiple": "2", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_split_rand_factor": "20", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_update_to": "1000", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_enable": "true", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-18T23:10:50.089 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_xfs_extsize": "false", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "filestore_zfs_snap": "false", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fio_dir": "/tmp/fio", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fsid": "b8d0ffd9-6668-442b-a039-a24cea734f3e", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_allow_other": "true", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_atomic_o_trunc": "true", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_big_writes": "true", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_debug": "false", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_default_permissions": "false", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_disable_pagecache": "false", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_max_write": "0", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_multithreaded": "true", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_require_active_mds": "true", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_set_user_groups": "true", 2024-04-18T23:10:50.090 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_splice_move": "true", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_splice_read": "true", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_splice_write": "true", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "fuse_use_invalidate_cb": "true", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "gss_target_name": "ceph", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_file": "", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_inject_failure": "0", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "heartbeat_interval": "5", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "host": "smithi073", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-18T23:10:50.091 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "inject_early_sigterm": "false", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "jaeger_agent_port": "6799", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "jaeger_tracing_enable": "false", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_aio": "true", 2024-04-18T23:10:50.092 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_align_min_size": "65536", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_block_align": "true", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_block_size": "4096", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_dio": "true", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_discard": "false", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_force_aio": "false", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_ignore_corruption": "false", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_max_write_bytes": "10485760", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_max_write_entries": "100", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_replay_from": "0", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_write_header_frequency": "0", 2024-04-18T23:10:50.093 INFO:teuthology.orchestra.run.smithi073.stdout: "journal_zero_on_create": "false", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "journaler_prefetch_periods": "10", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "journaler_prezero_periods": "5", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "journaler_write_head_interval": "15", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "keyfile": "", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_backend": "rocksdb", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_default_stripe_size": "65536", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_fsck_on_mount": "false", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_max_bytes": "67108864", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_max_ops": "512", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_nid_prealloc": "1024", 2024-04-18T23:10:50.094 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_onode_map_size": "1024", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_sync_submit_transaction": "false", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "kstore_sync_transaction": "false", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "librados_thread_count": "2", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "lockdep": "false", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "lockdep_force_backtrace": "false", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_coarse_timestamps": "true", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_flush_on_exit": "false", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_graylog_host": "127.0.0.1", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_graylog_port": "12201", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_max_new": "1000", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_max_recent": "10000", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_stderr_prefix": "", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_stop_at_utilization": "0.970000", 2024-04-18T23:10:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_file": "true", 2024-04-18T23:10:50.096 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_graylog": "false", 2024-04-18T23:10:50.096 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_journald": "false", 2024-04-18T23:10:50.096 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_stderr": "false", 2024-04-18T23:10:50.096 INFO:teuthology.orchestra.run.smithi073.stdout: "log_to_syslog": "false", 2024-04-18T23:10:50.096 INFO:teuthology.orchestra.run.smithi073.stdout: "max_rotating_auth_attempts": "10", 2024-04-18T23:10:50.096 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-18T23:10:50.096 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_action_on_write_error": "1", 2024-04-18T23:10:50.096 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_alternate_name_max": "8192", 2024-04-18T23:10:50.096 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_asio_thread_count": "2", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_export_pin": "true", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_dirs": "true", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_interval": "5", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_interval": "10", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_max": "-1", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_max_until": "-1", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_merge_size": "50", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_midchunk": "0.300000", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_min_start": "0.200000", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_minchunk": "0.001000", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_mode": "0", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_need_max": "1.200000", 2024-04-18T23:10:50.097 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_need_min": "0.800000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_overload_epochs": "2", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_bits": "3", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_size": "10000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_target_decay": "10.000000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_beacon_grace": "15.000000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_beacon_interval": "4.000000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_mid": "0.700000", 2024-04-18T23:10:50.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_delay": "0", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_sleep": "200", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_splitauth": "true", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_quiesce_threshold": "524288", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_release_free_interval": "10", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_reservation": "0.050000", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_trim_interval": "1", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cache_trim_threshold": "262144", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-18T23:10:50.099 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_client_prealloc_inos": "1000", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_connect_bootstrapping": "false", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_damage_table_max_entries": "10000", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_auth_pins": "false", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_frag": "false", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_scatterstat": "false", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_debug_subtrees": "false", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_decay_halflife": "5.000000", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_default_dir_hash": "2", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_defer_session_stale": "true", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_deny_all_reconnect": "false", 2024-04-18T23:10:50.100 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_keys_per_op": "16384", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_max_commit_size": "10", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_max_entries": "0", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dir_prefetch": "true", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_on_map": "false", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_early_reply": "true", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_enable_op_tracker": "true", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_enforce_unique_name": "true", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_random": "true", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-18T23:10:50.101 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_health_summarize_threshold": "10", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_health_dummy": "false", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_migrator_session_race": "false", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-18T23:10:50.102 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_join_fs": "", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_journal_format": "1", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_create_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_export_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_import_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_journal_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_journal_expire_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_journal_replay_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_link_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_mdstable_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_openc_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_rename_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_shutdown_at": "0", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_event_large_threshold": "524288", 2024-04-18T23:10:50.103 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_events_per_segment": "1024", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_major_segment_event_ratio": "12", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_max_events": "-1", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_max_segments": "128", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_pause": "false", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_segment_size": "0", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_skip_unbounded_events": "false", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_trim_threshold": "128", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_log_warn_factor": "2.000000", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_caps_per_client": "1048576", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_completed_flushes": "100000", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_completed_requests": "100000", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_export_size": "20971520", 2024-04-18T23:10:50.104 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_file_recover": "32", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_purge_files": "64", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_purge_ops": "8192", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_max_snaps_per_dir": "100", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_metrics_update_interval": "2", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_min_caps_per_client": "100", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_min_caps_working_set": "10000", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_numa_node": "-1", 2024-04-18T23:10:50.105 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_complaint_time": "30.000000", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_duration": "600", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_size": "20", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_slow_op_size": "20", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_op_log_threshold": "5", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_ping_grace": "15", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_ping_interval": "5", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_max_caps": "30000", 2024-04-18T23:10:50.106 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_recall_warning_threshold": "262144", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_replay_interval": "1.000000", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_root_ino_gid": "0", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_root_ino_uid": "0", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-18T23:10:50.107 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_shutdown_check": "0", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_skip_ino": "0", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_snap_max_uid": "4294967294", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_snap_min_uid": "0", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_snap_rstat": "false", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_standby_replay_damaged": "false", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_symlink_recovery": "true", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-18T23:10:50.108 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_thrash_exports": "0", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_thrash_fragments": "0", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_tick_interval": "5.000000", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_valgrind_exit": "false", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_verify_backtrace": "1", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_verify_scatter": "false", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mds_wipe_sessions": "false", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mempool_debug": "false", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_device_bytes": "1073741824", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_page_set": "false", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "memstore_page_size": "65536", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_client_bytes": "134217728", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_client_messages": "512", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-18T23:10:50.109 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_disabled_modules": "", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_max_pg_num_change": "128", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mds_bytes": "134217728", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mds_messages": "128", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mon_bytes": "134217728", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_mon_messages": "128", 2024-04-18T23:10:50.110 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_osd_bytes": "536870912", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_osd_messages": "8192", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_pool": "true", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_standby_modules": "true", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_stats_period": "5", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_stats_threshold": "5", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_tick_period": "2", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_allow_pool_delete": "true", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_allow_pool_size_one": "true", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_auth_validate_all_caps": "true", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-18T23:10:50.111 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_bytes": "104857600", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_directed_command_retry": "2", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_hunt_parallel": "3", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_log_interval": "1.000000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_ping_interval": "10.000000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_client_target_rank": "-1", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-18T23:10:50.112 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_file_level": "debug", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_file": "true", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_journald": "false", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_compact_on_bootstrap": "false", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_compact_on_start": "false", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_compact_on_trim": "true", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-18T23:10:50.113 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_cpu_threads": "4", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_crush_min_required_version": "hammer", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_daemon_bytes": "419430400", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data_avail_crit": "5", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data_avail_warn": "5", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_data_size_warn": "16106127360", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_dump_json": "false", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_dump_transactions": "false", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_extra_checks": "false", 2024-04-18T23:10:50.114 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_require_reef": "false", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_no_require_squid": "false", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_down_mkfs_grace": "120", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_down_uptime_grace": "60", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_election_default_strategy": "3", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_election_timeout": "5.000000", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_elector_ping_divisor": "2", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_enable_op_tracker": "true", 2024-04-18T23:10:50.115 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_fake_pool_delete": "false", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_force_quorum_join": "false", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_globalid_prealloc": "10000", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_detail_to_clog": "false", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_log_update_period": "5", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_max_detail": "50", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_to_clog": "true", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_to_clog_interval": "600", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_host": "172.21.15.73,172.21.15.120,172.21.15.136", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_host_override": "", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_initial_members": "", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-18T23:10:50.116 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_lease": "5.000000", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_log_full_interval": "50", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_log_max": "10000", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_log_max_summary": "50", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_log_epochs": "500", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_osd": "10000", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_pg_per_osd": "10000", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_pool_pg_num": "65536", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-18T23:10:50.117 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mds_force_trim_to": "0", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mds_skip_sanity": "false", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_memory_autotune": "true", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_memory_target": "2147483648", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_beacon_grace": "30", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_digest_period": "5", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_inactive_grace": "60", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_min_osdmap_epochs": "500", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_complaint_time": "30", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_duration": "600", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_size": "20", 2024-04-18T23:10:50.118 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_slow_op_size": "20", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_op_log_threshold": "5", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_auto_mark_in": "false", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_cache_size": "500", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_down_out_interval": "600", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-18T23:10:50.119 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_force_trim_to": "0", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_max_creating_pgs": "1024", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_min_down_reporters": "2", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-18T23:10:50.120 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_report_timeout": "900", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_stuck_threshold": "60", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-18T23:10:50.121 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_probe_timeout": "2.000000", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_max_change": "0.050000", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_max_osds": "4", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_interval": "300", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_max_keys": "100", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_scrub_timeout": "300", 2024-04-18T23:10:50.122 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_session_timeout": "300", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_smart_report_timeout": "5", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stat_smooth_intervals": "6", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_pool_min_size": "2", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_pool_size": "4", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_debug": "false", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_provider_kill_at": "0", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_requester_kill_at": "0", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_sync_timeout": "60.000000", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_target_pg_per_osd": "100", 2024-04-18T23:10:50.123 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_tick_interval": "5", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_timecheck_interval": "300.000000", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_use_min_delay_socket": "false", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_older_version_delay": "604800", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_misplaced": "false", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_older_version": "true", 2024-04-18T23:10:50.124 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_pool_no_app": "true", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "monmap": "", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-04-18T23:10:50.125 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "motr_tracing_enabled": "false", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_op_threads": "3", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_cm": "false", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_device_name": "", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_dscp": "96", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_local_gid": "", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_port_num": "1", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-18T23:10:50.126 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_sl": "3", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_support_srq": "true", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_rdma_type": "ib", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_async_reap_threshold": "5", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_before_connect": "false", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_ipv4": "true", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_ipv6": "false", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_msgr1": "true", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_msgr2": "true", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_port_max": "7568", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_port_min": "6800", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_retry_count": "3", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_bind_retry_delay": "5", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_client": "false", 2024-04-18T23:10:50.127 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_mds": "false", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_mgr": "false", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_mon": "false", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_blackhole_osd": "false", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_client_mode": "crc secure", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_cluster_mode": "crc secure", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_cluster_type": "", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_compress_secure": "false", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_connection_idle_timeout": "900", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_connection_ready_timeout": "10", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_crc_data": "true", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_crc_header": "true", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_bad_msg": "false", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_bug": "true", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_old_message": "true", 2024-04-18T23:10:50.128 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_skipped_message": "false", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_coremask": "0xF", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_enable_tso": "true", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_hugepages": "", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_lro": "true", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_memory_channel": "4", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-18T23:10:50.129 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_pmd": "", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_port_id": "0", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_dump_on_send": "false", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_initial_backoff": "0.200000", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_delay_max": "1.000000", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_delay_type": "", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_network_congestion": "0", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_inject_socket_failures": "0", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_learn_addr_from_peer": "true", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_max_accept_failures": "4", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_max_backoff": "15.000000", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_mon_client_mode": "secure crc", 2024-04-18T23:10:50.130 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_mon_service_mode": "secure crc", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_osd_compress_min_size": "1024", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_osd_compress_mode": "none", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_osd_compression_algorithm": "snappy", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_pq_min_cost": "65536", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_public_type": "", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_service_mode": "crc secure", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_listen_backlog": "512", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_nodelay": "true", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_tcp_rcvbuf": "0", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "ms_type": "async+posix", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "no_config_file": "false", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_completion_locks_per_session": "32", 2024-04-18T23:10:50.131 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_inflight_ops": "1024", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_tick_interval": "5.000000", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "objecter_timeout": "10.000000", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "objectstore_blackhole": "false", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "openssl_engine_opts": "", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_delay_time": "5.000000", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_hist_halflife": "1000", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_max_low_ops": "2", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_max_ops": "4", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-18T23:10:50.132 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_agent_slop": "0.020000", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_asio_thread_count": "2", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_async_recovery_min_cost": "1", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backfill_scan_max": "512", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backfill_scan_min": "64", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backoff_on_degraded": "false", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backoff_on_peering": "false", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_backoff_on_unfound": "true", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_beacon_report_interval": "300", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_duration": "30", 2024-04-18T23:10:50.133 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_max_block_size": "67108864", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_blkin_trace_all": "false", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_blocked_scrub_grace_period": "120", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_check_for_log_corruption": "false", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_dir": "/usr/lib/rados-classes", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_class_update_on_start": "true", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_message_cap": "256", 2024-04-18T23:10:50.134 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_message_size_cap": "524288000", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_op_priority": "63", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_client_watch_timeout": "30", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_command_max_records": "256", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_compact_on_start": "false", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_update_on_start": "true", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_crush_update_weight_set": "true", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-18T23:10:50.135 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_feed_pullee": "-1", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_inject_dispatch_delay_probability": "0.100000", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_misdirected_ops": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_no_acting_change": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_no_purge_strays": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_op_order": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_shutdown": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-18T23:10:50.136 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_trim_objects": "false", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_snaps": "false", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_keys": "1024", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_stride": "524288", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_default_notify_timeout": "30", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep": "0.000000", 2024-04-18T23:10:50.137 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_discard_disconnected_ops": "true", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_enable_op_tracker": "true", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_extblkdev_plugins": "vdo", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_info": "true", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_shutdown": "true", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-18T23:10:50.138 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_function_tracing": "false", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_grace": "20", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_interval": "6", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_min_peers": "10", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_min_size": "2000", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_stale": "600", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_hit_set_max_size": "100000", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_hit_set_min_size": "1000", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-18T23:10:50.139 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_journal_size": "5120", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_kill_backfill_at": "0", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_cache_size": "50", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_dedup": "true", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_message_max": "40", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_map_share_max_epochs": "40", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_attr_name_len": "100", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_attr_size": "0", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_backfills": "1", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_markdown_count": "5", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_markdown_period": "600", 2024-04-18T23:10:50.140 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_object_name_len": "2048", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_object_namespace_len": "256", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_object_size": "134217728", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pg_blocked_by": "16", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pg_log_entries": "10000", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_pgls": "1024", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_push_cost": "8388608", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_push_objects": "10", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_scrubs": "3", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_trimming_pgs": "2", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_write_op_reply_len": "64", 2024-04-18T23:10:50.141 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_max_write_size": "90", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_profile": "balanced", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-18T23:10:50.142 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_base": "805306368", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_cache_min": "134217728", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_target": "4294967296", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_target_autotune": "false", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_min_pg_log_entries": "250", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_min_recovery_priority": "0", 2024-04-18T23:10:50.143 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_report_interval": "5", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_num_cache_shards": "32", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_num_op_tracker_shard": "32", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_numa_auto_affinity": "true", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_numa_node": "-1", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_numa_prefer_iface": "true", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objecter_finishers": "1", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore": "bluestore", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore_fuse": "false", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_objectstore_tracing": "false", 2024-04-18T23:10:50.144 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_complaint_time": "30.000000", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_duration": "600", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_size": "20", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_slow_op_size": "20", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_log_threshold": "5", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_shards": "0", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_shards_hdd": "5", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_shards_ssd": "8", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_pq_min_cost": "65536", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-18T23:10:50.145 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_queue_cut_off": "high", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_op_thread_timeout": "15", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_open_classes_on_start": "true", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_os_flags": "0", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_peering_op_priority": "255", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_delete_cost": "1048576", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_delete_priority": "5", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_log_trim_max": "10000", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_log_trim_min": "100", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-18T23:10:50.146 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_crimson": "false", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-18T23:10:50.147 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_flags": "0", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_min_size": "0", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_pg_num": "32", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_pgp_num": "0", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_read_ratio": "70", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_size": "2", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_default_type": "replicated", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_push_per_object_cost": "1000", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-18T23:10:50.148 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recover_clone_overlap": "true", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_cost": "20971520", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_active": "0", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_max_single_start": "1", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_op_priority": "3", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_priority": "5", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-18T23:10:50.149 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep": "0.000000", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_repair_during_recovery": "false", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_requested_scrub_priority": "120", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_auto_repair": "false", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_begin_hour": "0", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_begin_week_day": "0", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_chunk_max": "25", 2024-04-18T23:10:50.150 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_chunk_min": "5", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_cost": "52428800", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_disable_reservation_queuing": "true", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_during_recovery": "false", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_end_hour": "0", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_end_week_day": "0", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_event_cost": "4096", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_invalid_stats": "true", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_max_preemptions": "5", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_priority": "5", 2024-04-18T23:10:50.151 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_reservation_timeout": "300000", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_sleep": "0.000000", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_scrub_slow_reservation_response": "30000", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_skip_data_digest": "false", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_smart_report_timeout": "5", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_cost": "1048576", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_priority": "5", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-04-18T23:10:50.152 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_stats_update_period_scrubbing": "15", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_target_transaction_size": "30", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_tracing": "false", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_use_stale_snap": "false", 2024-04-18T23:10:50.153 INFO:teuthology.orchestra.run.smithi073.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "osdc_blkin_trace_all": "false", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_kill_at": "0", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_max_join_drift": "10", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_min": "500", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_min_wait": "0.050000", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_propose_interval": "1.000000", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_service_trim_max": "500", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_service_trim_min": "250", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_stash_full_interval": "25", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_trim_max": "500", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "paxos_trim_min": "250", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "perf": "true", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-18T23:10:50.154 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "plugin_dir": "/usr/lib/ceph", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addr": "-", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "public_addrv": "", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "public_bind_addr": "-", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "public_network": "", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "public_network_interface": "", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "qat_compressor_enabled": "false", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "qat_compressor_session_max_number": "256", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "rados_mon_op_timeout": "0", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "rados_osd_op_timeout": "0", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "rados_tracing": "false", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_atime_update_interval": "60", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_balance_parent_reads": "false", 2024-04-18T23:10:50.155 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_balance_snap_reads": "false", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_blkin_trace_all": "false", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache": "true", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_policy": "writearound", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_size": "33554432", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_clone_copy_on_read": "false", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_compression_hint": "none", 2024-04-18T23:10:50.156 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_concurrent_management_ops": "10", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_clone_format": "auto", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_data_pool": "", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_features": "61", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_format": "2", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_map_options": "", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_order": "22", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_pool": "rbd", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_stripe_count": "0", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_default_stripe_unit": "0", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-18T23:10:50.157 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_enable_alloc_hint": "true", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_io_scheduler": "simple", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_order": "24", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_pool": "", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_journal_splay_width": "4", 2024-04-18T23:10:50.158 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_localize_parent_reads": "false", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_localize_snap_reads": "false", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-18T23:10:50.159 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-18T23:10:50.160 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_mtime_update_interval": "60", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_non_blocking_aio": "true", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_op_thread_timeout": "60", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_op_threads": "1", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_parent_cache_enabled": "false", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_plugins": "", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_bps_burst": "0", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_bps_limit": "0", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_exclude_ops": "0", 2024-04-18T23:10:50.161 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_iops_burst": "0", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_iops_limit": "0", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-18T23:10:50.162 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_read_from_replica_policy": "default", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_skip_partial_discard": "true", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_tracing": "false", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_validate_names": "true", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rbd_validate_pool": "true", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "restapi_base_url": "", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "restapi_log_level": "", 2024-04-18T23:10:50.163 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_acl_grants_max_num": "100", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_admin_entry": "admin", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_backend_store": "rados", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_barbican_url": "", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_beast_enable_async": "true", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_counters_cache": "false", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-04-18T23:10:50.164 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cache_enabled": "true", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cache_expiry_interval": "900", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cache_lru_size": "10000", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_config_store": "rados", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_content_length_compat": "false", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_copy_obj_progress": "true", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cors_rules_max_num": "100", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_cross_domain_policy": "", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_addr": "", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-18T23:10:50.165 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_password": "", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_kmip_username": "", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_require_ssl": "true", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-18T23:10:50.166 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_addr": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_auth": "token", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_namespace": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_prefix": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_token_file": "", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-18T23:10:50.167 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_buffersize": "524288", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_low_speed_time": "300", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_tcp_keepalive": "0", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_host": "127.0.0.1", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_d4n_port": "6379", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-18T23:10:50.168 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_changes_size": "1000", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_num_shards": "128", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_log_window": "30", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_notify_interval_msec": "0", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_sync_poll_interval": "20", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_data_sync_spawn_window": "20", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-18T23:10:50.169 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dns_name": "", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dns_s3website_name": "", 2024-04-18T23:10:50.170 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_dynamic_resharding": "true", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_gc_threads": "true", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_lc_threads": "true", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_ops_log": "false", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_quota_threads": "true", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_static_website": "false", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enable_usage_log": "false", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_enforce_swift_acls": "true", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_exit_timeout_secs": "120", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_expose_bucket": "false", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_extended_http_attrs": "", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_filter": "none", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_frontends": "beast port=7480", 2024-04-18T23:10:50.171 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_deferred": "50", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_objs": "32", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_gc_processor_period": "3600", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_init_timeout": "300", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-04-18T23:10:50.172 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_kafka_connection_idle": "300", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_kafka_message_timeout": "5000", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_kafka_sleep_timeout": "10", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_domain": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_password": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_password_path": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_project": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_tenant": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_token": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_token_path": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_admin_user": "", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_api_version": "2", 2024-04-18T23:10:50.173 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_domain": "", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_password": "", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_project": "", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_barbican_user": "", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_url": "", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_debug_interval": "-1", 2024-04-18T23:10:50.174 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_lock_max_time": "90", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_objs": "32", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_rules": "1000", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_worker": "3", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lc_thread_delay": "0", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_dnattr": "uid", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_searchfilter": "", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ldap_uri": "", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-18T23:10:50.175 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_http_headers": "", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_log_object_name_utc": "false", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_attr_name_len": "0", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_attr_size": "0", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_chunk_size": "4194304", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_listing_results": "1000", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_notify_retries": "10", 2024-04-18T23:10:50.176 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_put_param_size": "1048576", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_put_size": "5368709120", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_max_slo_entries": "1000", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_md_log_max_shards": "64", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_meta_sync_poll_interval": "20", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_meta_sync_spawn_window": "20", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_mp_lock_max_time": "600", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-18T23:10:50.177 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_max_gc": "300", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_num_async_rados_threads": "32", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_num_control_oids": "8", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_numa_node": "-1", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-18T23:10:50.178 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_objexp_chunk_size": "100", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_objexp_gc_interval": "600", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_op_thread_timeout": "600", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_op_tracing": "false", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_opa_token": "", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_opa_url": "", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_opa_verify_ssl": "true", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_rados": "false", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_ops_log_socket_path": "", 2024-04-18T23:10:50.179 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_push_interval": "2.000000", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_cache_lanes": "3", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_cache_max_buckets": "100", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_cache_partitions": "3", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_print_continue": "true", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-18T23:10:50.180 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rados_tracing": "false", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_realm": "", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_realm_id": "", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_region": "", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_request_uri": "", 2024-04-18T23:10:50.181 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_batch_size": "64", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_max_aio": "128", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_num_logs": "16", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_reshard_thread_interval": "600", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_resolve_cname": "false", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_run_sync_thread": "true", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-18T23:10:50.182 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_scheduler_type": "throttler", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_script_uri": "", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_service_provider_name": "", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_client_id": "", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_client_secret": "", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_entry": "sts", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_key": "sts", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_min_session_duration": "900", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sts_token_introspection_url": "", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_account_in_url": "false", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_auth_entry": "auth", 2024-04-18T23:10:50.183 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_auth_url": "", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_custom_header": "", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_need_stats": "true", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_tenant_name": "", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_token_expiration": "86400", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_url": "", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_url_prefix": "swift", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_lease_period": "120", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-18T23:10:50.184 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_thread_pool_size": "512", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_topic_persistency_max_retries": "0", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_topic_require_publish_policy": "false", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_comment": "", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_createby": "", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_encoding": "", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_flag": "false", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_max_size": "5368709120", 2024-04-18T23:10:50.185 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_origin": "", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_torrent_tracker": "", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_trust_forwarded_https": "false", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_max_shards": "32", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_usage_max_user_shards": "1", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_use_opa_authz": "false", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_counters_cache": "false", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_counters_cache_size": "10000", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_max_buckets": "1000", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_policies_max_num": "100", 2024-04-18T23:10:50.186 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_user_unique_email": "true", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_verify_ssl": "true", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zone": "", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zone_id": "", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zonegroup": "", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zonegroup_id": "", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgwlc_auto_session_clear": "true", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rgwlc_skip_bucket_step": "false", 2024-04-18T23:10:50.187 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_block_size": "4096", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_size": "536870912", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_index_type": "binary_search", 2024-04-18T23:10:50.188 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_partition_filters": "false", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_perf": "false", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "run_dir": "/var/run/ceph", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_block_create": "true", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_cache_lru_size": "67108864", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_cbjournal_size": "5368709120", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_data_delta_based_overwrite": "0", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_default_max_object_size": "16777216", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_device_size": "53687091200", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_batch_capacity": "16", 2024-04-18T23:10:50.189 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_main_device_type": "SSD", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_max_concurrent_transactions": "8", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "seastore_segment_size": "67108864", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "set_keepcaps": "false", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "setgroup": "", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "setuser": "", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "setuser_match_path": "", 2024-04-18T23:10:50.190 INFO:teuthology.orchestra.run.smithi073.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-18T23:10:50.191 INFO:teuthology.orchestra.run.smithi073.stdout: "thp": "false", 2024-04-18T23:10:50.191 INFO:teuthology.orchestra.run.smithi073.stdout: "threadpool_default_timeout": "60", 2024-04-18T23:10:50.191 INFO:teuthology.orchestra.run.smithi073.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-18T23:10:50.191 INFO:teuthology.orchestra.run.smithi073.stdout: "throttler_perf_counter": "true" 2024-04-18T23:10:50.191 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:10:50.193 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-18T23:10:50.326 INFO:teuthology.orchestra.run.smithi073.stdout:mon.a: {} 2024-04-18T23:10:50.327 INFO:teuthology.orchestra.run.smithi073.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T23:10:50.339 INFO:teuthology.orchestra.run.smithi073.stdout:mon.b: {} 2024-04-18T23:10:50.339 INFO:teuthology.orchestra.run.smithi073.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T23:10:50.352 INFO:teuthology.orchestra.run.smithi073.stdout:mon.c: {} 2024-04-18T23:10:50.352 INFO:teuthology.orchestra.run.smithi073.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T23:10:50.367 DEBUG:teuthology.orchestra.run.smithi142:> type ceph-objectstore-tool 2024-04-18T23:10:50.373 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-18T23:10:50.373 DEBUG:teuthology.orchestra.run.smithi073:> type ceph-objectstore-tool 2024-04-18T23:10:50.377 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-18T23:10:50.378 DEBUG:teuthology.orchestra.run.smithi120:> type ceph-objectstore-tool 2024-04-18T23:10:50.383 INFO:teuthology.orchestra.run.smithi120.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-18T23:10:50.383 DEBUG:teuthology.orchestra.run.smithi136:> type ceph-objectstore-tool 2024-04-18T23:10:50.388 INFO:teuthology.orchestra.run.smithi136.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-18T23:10:50.388 INFO:teuthology.run_tasks:Running task rados... 2024-04-18T23:10:50.399 INFO:tasks.rados:Beginning rados... 2024-04-18T23:10:50.399 DEBUG:teuthology.run_tasks:Unwinding manager rados 2024-04-18T23:10:50.407 INFO:tasks.rados:joining rados 2024-04-18T23:10:50.407 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-18T23:10:50.407 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-18T23:10:50.407 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-18T23:10:50.408 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-18T23:10:50.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-18T23:10:50.408 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-18T23:10:50.409 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-18T23:10:50.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:10:50.409 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-18T23:10:50.409 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:10:50.410 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-18T23:10:50.410 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T23:10:50.410 INFO:tasks.rados:clients are ['client.0'] 2024-04-18T23:10:50.410 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set shecprofile c=2 crush-failure-domain=osd k=4 m=3 name=shecprofile plugin=shec 2024-04-18T23:10:50.426 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:50.425+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:50.510 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:50.509+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:50.536 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:10:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:10:50.537 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:10:50.537 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:10:50.555 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:10:50.558 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:10:50.558 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:10:50.571 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:50.571 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:50.571 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:50.571 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:50.571 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:50.571 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:50.571 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:50.571 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:50.571 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:50.574 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:50.574 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:50.574 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-18T23:10:50.581 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:10:50.581 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:10:50.612 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:10:50.612 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:10:50.612 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:10:50.610+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:10:50.635 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:10:50.636 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:10:50.654 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:10:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:10:50.673 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:50.716 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:50.717 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:10:50.718 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:10:50.719 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:10:50.719 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:10:50.720 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:10:50.720 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:10:50.720 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:10:50.720 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:50.719+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:10:50.731 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:10:50.731 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:10:50.754 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:10:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:10:50.789 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:10:50.789 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:10:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:50.800 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:50.800 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:50.800 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-18T23:10:50.810 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:10:50.811 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:10:50.818 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:50.819+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:10:50.843 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:10:50.843 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:10:50.874 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:50.874 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:50.876 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:10:50.876 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:10:50.888 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:10:50.897 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:10:50.898 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:10:50.917 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:10:50.918 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:10:50.919 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:10:50.917+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:10:50.953 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:50.953 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:50.953 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:50.953 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:50.953 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:50.953 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:50.954 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:50.956 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:50.956 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:50.957 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-18T23:10:51.005 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:51.005 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:51.019 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:10:51.020 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:10:51.017+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:10:51.120 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:51.118+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:10:51.167 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:51.167 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:51.180 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:10:51.224 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:10:51.222+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:10:51.250 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:51.250 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:51.250 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:51.250 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:51.250 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:51.250 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:51.250 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:51.250 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:51.250 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:51.251 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:51.251 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:51.251 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:51.253 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:51.253 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:51.254 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-04-18T23:10:51.323 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:51.321+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:10:51.331 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:51.331 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:51.344 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:10:51.418 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-04-18T23:10:51.418 INFO:teuthology.orchestra.run.smithi073.stderr:load: shec 2024-04-18T23:10:51.424 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:51.421+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:10:51.441 INFO:tasks.rados:starting run 0 out of 1 2024-04-18T23:10:51.441 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2024-04-18T23:10:51.441 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 16 erasure shecprofile 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:51.461 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:51.463 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:51.464 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:51.464 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-04-18T23:10:51.509 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:51.509 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:51.524 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:51.523+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:10:51.525 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:10:51.625 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:51.622+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:10:51.678 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:51.678 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:51.691 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:51.706 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:51.707 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:51.707 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-04-18T23:10:51.726 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:51.727+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:10:51.808 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:51.809 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:51.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:10:51.827 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:51.827+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:10:51.890 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:51.890 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:51.890 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:51.890 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:51.890 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:51.890 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:51.890 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:51.891 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:51.891 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:51.891 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:51.891 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:51.891 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:51.893 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:51.893 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:51.894 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-04-18T23:10:51.928 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:10:51.925+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:10:51.936 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:51.936 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:51.949 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:10:52.030 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:10:52.031+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:10:52.031 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:52.032 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:52.034 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:52.035 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:52.035 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 'trim stale osdmaps' 2024-04-18T23:10:52.098 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:52.098 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:52.112 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:10:52.131 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:52.129+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:52.219 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:52.221 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:52.222 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:52.222 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 'trim stale osdmaps' 2024-04-18T23:10:52.231 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:10:52.231+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:10:52.264 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:52.264 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:52.277 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:10:52.332 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:10:52.330+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:10:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:52.357 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:52.358 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:52.360 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:52.360 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:52.360 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 'trim stale osdmaps' 2024-04-18T23:10:52.410 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:52.410 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:52.427 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:10:52.473 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:52.473+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:52.534 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:10:52.531+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:52.544 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:52.548 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:52.548 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:52.548 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 'trim stale osdmaps' 2024-04-18T23:10:52.558 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:10:52.634 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:52.633+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:52.683 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:52.686 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:52.686 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:52.686 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 'trim stale osdmaps' 2024-04-18T23:10:52.728 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:52.728 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:52.735 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:52.733+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:10:52.742 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:10:52.821 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:52.821 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:52.822 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:52.824 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:52.825 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:52.825 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 'trim stale osdmaps' 2024-04-18T23:10:52.867 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:52.867 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:52.868 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:52.866+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:10:52.880 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:10:52.936 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:52.933+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:10:52.937 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:10:52.964 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:52.964 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:52.964 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:52.964 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:52.964 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:52.964 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:52.964 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:52.964 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:52.965 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:52.965 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:52.965 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:52.965 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:52.967 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:52.967 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:52.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 'trim stale osdmaps' 2024-04-18T23:10:53.002 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:53.002 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:53.016 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:10:53.037 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:53.033+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:10:53.062 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:10:53.062 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:10:53.080 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:10:53.081 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:10:53.101 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:10:53.101 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:10:53.121 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:10:53.121 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:10:53.138 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:53.137+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:10:53.152 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:53.152 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:53.152 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:53.152 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:53.153 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:53.153 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:53.153 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:53.153 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:53.153 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:53.153 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:53.153 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:53.153 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:53.154 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:10:53.155 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:10:53.155 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:53.156 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:53.156 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 'trim stale osdmaps' 2024-04-18T23:10:53.238 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:53.237+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:53.244 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:53.244 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:53.259 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:10:53.263 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:10:53.264 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:10:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:10:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:10:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:10:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:10:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:10:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:10:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:10:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:10:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:10:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:10:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:10:53.297 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:10:53.299 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:10:53.299 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:10:53.339 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:53.337+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:53.370 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:10:53.370 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:10:53.418 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:53.418 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:53.429 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'unique_pool_0' created 2024-04-18T23:10:53.435 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:10:53.441 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:10:53.439+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:10:53.447 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T23:10:53.453 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2024-04-18T23:10:53.488 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:10:53.489 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:10:53.542 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:53.538+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:10:53.560 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:53.561 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:53.574 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:10:53.589 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:10:53.590 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:10:53.642 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:53.641+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:53.711 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:53.711 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:53.724 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:10:53.724 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:10:53.725 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:10:53.742 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:53.743+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:10:53.842 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:10:53.842 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:10:53.844 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:53.843+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:10:53.890 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:53.890 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:53.904 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:10:53.950 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:10:53.949+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:10:53.961 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:10:53.961 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:10:54.026 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:54.027 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:54.041 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:10:54.045 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:10:54.045 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:10:54.046 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:10:54.047+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:10:54.147 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:10:54.147+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:10:54.167 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:10:54.167 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:10:54.216 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:10:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:10:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:10:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:10:54.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-18T23:10:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:10:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.836584584000001, 2024-04-18T23:10:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50897375199999995, 2024-04-18T23:10:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.217 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1 2024-04-18T23:10:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:10:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:10:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377587+0000", 2024-04-18T23:10:54.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3089999999999998e-06 2024-04-18T23:10:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377601+0000", 2024-04-18T23:10:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3862999999999999e-05 2024-04-18T23:10:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377623+0000", 2024-04-18T23:10:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1741000000000001e-05 2024-04-18T23:10:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377644+0000", 2024-04-18T23:10:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1024000000000001e-05 2024-04-18T23:10:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.576350+0000", 2024-04-18T23:10:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198706355 2024-04-18T23:10:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:10:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.576365+0000", 2024-04-18T23:10:54.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4748e-05 2024-04-18T23:10:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.682700+0000", 2024-04-18T23:10:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106335283 2024-04-18T23:10:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.683098+0000", 2024-04-18T23:10:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039820900000000001 2024-04-18T23:10:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.683616+0000", 2024-04-18T23:10:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051731399999999995 2024-04-18T23:10:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.688337+0000", 2024-04-18T23:10:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047211509999999998 2024-04-18T23:10:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886392+0000", 2024-04-18T23:10:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198055008 2024-04-18T23:10:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886494+0000", 2024-04-18T23:10:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102203 2024-04-18T23:10:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886559+0000", 2024-04-18T23:10:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4543999999999997e-05 2024-04-18T23:10:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:10:54.227 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6199095970000004, 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41468469299999999, 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 110 2024-04-18T23:10:54.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:10:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:10:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594261+0000", 2024-04-18T23:10:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:10:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594266+0000", 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9690000000000002e-06 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594278+0000", 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2048999999999999e-05 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594286+0000", 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2900000000000002e-06 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.804556+0000", 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.21026968800000001 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.804784+0000", 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022826000000000001 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.805113+0000", 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032918500000000001 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.908604+0000", 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103490558 2024-04-18T23:10:54.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008799+0000", 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100195153 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008880+0000", 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1329999999999996e-05 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008945+0000", 2024-04-18T23:10:54.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4400999999999995e-05 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6198718269999999, 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41616193699999998, 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.235 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 111 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6300000000000004e-07 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594305+0000", 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594317+0000", 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1786e-05 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594327+0000", 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7249999999999992e-06 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906203+0000", 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.31187656499999999 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906336+0000", 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013266800000000001 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906542+0000", 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020588900000000001 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009954+0000", 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10341181100000001 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010185+0000", 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023115700000000001 2024-04-18T23:10:54.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010324+0000", 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013910800000000001 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010460+0000", 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000135635 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6198310940000002, 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42114173300000002, 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.242 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 112 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594339+0000", 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7700000000000004e-07 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594484+0000", 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000144804 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594500+0000", 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5752000000000001e-05 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594511+0000", 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1039000000000001e-05 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907178+0000", 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.31266760199999999 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907320+0000", 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014133099999999999 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907545+0000", 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022546 2024-04-18T23:10:54.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010139+0000", 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102593936 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015192+0000", 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050530590000000004 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015364+0000", 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017223600000000001 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015480+0000", 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011573700000000001 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.248 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6196422970000004, 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42137598300000001, 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 113 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594528+0000", 2024-04-18T23:10:54.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594533+0000", 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0130000000000001e-06 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594545+0000", 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2009e-05 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594553+0000", 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7389999999999999e-06 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009008+0000", 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41445484999999999 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009125+0000", 2024-04-18T23:10:54.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117773 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009350+0000", 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022510400000000001 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015618+0000", 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062671189999999998 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015662+0000", 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4619000000000001e-05 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015798+0000", 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013600600000000001 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015903+0000", 2024-04-18T23:10:54.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104941 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6196049769999998, 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62069844100000005, 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.255 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 114 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594565+0000", 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594570+0000", 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.186e-06 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594582+0000", 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1763000000000001e-05 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594590+0000", 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5290000000000003e-06 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010572+0000", 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.415982252 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010727+0000", 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015558400000000001 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010971+0000", 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000243339 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015929+0000", 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049583639999999998 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215108+0000", 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19917934400000001 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215199+0000", 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0556000000000005e-05 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215263+0000", 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3874000000000005e-05 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.261 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6195686499999997, 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62089894099999998, 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 115 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.262 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594746+0000", 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014446700000000001 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594761+0000", 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4792e-05 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594769+0000", 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6160000000000002e-06 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013742+0000", 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.418972438 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013974+0000", 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023265699999999999 2024-04-18T23:10:54.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.014328+0000", 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035345 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018069+0000", 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037411079999999999 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215346+0000", 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19727757900000001 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215435+0000", 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8245999999999995e-05 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215500+0000", 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4968000000000005e-05 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6193853990000004, 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.72133309899999998, 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.268 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 116 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594785+0000", 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594790+0000", 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4229999999999996e-06 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594803+0000", 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2196e-05 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594810+0000", 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6190000000000002e-06 2024-04-18T23:10:54.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016062+0000", 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42125235 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016215+0000", 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000152051 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016420+0000", 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000205916 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018082+0000", 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001661718 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.315968+0000", 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29788616600000001 2024-04-18T23:10:54.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316051+0000", 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2897000000000003e-05 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316117+0000", 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5866999999999998e-05 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6193477659999997, 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.721588585, 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.275 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 117 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594829+0000", 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7830000000000001e-06 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594841+0000", 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1505000000000001e-05 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.694940+0000", 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100099585 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016967+0000", 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32202692799999999 2024-04-18T23:10:54.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017224+0000", 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000256548 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017583+0000", 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000359689 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.020139+0000", 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025556649999999999 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316256+0000", 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29611688800000002 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316345+0000", 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9285999999999999e-05 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316410+0000", 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5058e-05 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074239+0000", 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.1399307350000001, 2024-04-18T23:10:54.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40991988800000001, 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 135 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074239+0000", 2024-04-18T23:10:54.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074239+0000", 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074240+0000", 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0130000000000001e-06 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074245+0000", 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4360000000000001e-06 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074254+0000", 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2920000000000001e-06 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074262+0000", 2024-04-18T23:10:54.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4499999999999998e-06 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.077522+0000", 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003260144 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.077589+0000", 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6827000000000002e-05 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.077694+0000", 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010524200000000001 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080701+0000", 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030069039999999999 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484009+0000", 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40330789299999997 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484085+0000", 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6299999999999998e-05 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484159+0000", 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3386999999999994e-05 2024-04-18T23:10:54.287 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074279+0000", 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.1398906090000001, 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41014821299999998, 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.288 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 136 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074279+0000", 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074279+0000", 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074279+0000", 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5300000000000004e-07 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074284+0000", 2024-04-18T23:10:54.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9439999999999996e-06 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074294+0000", 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6919999999999997e-06 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074301+0000", 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.233e-06 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.078022+0000", 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037207429999999999 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.078091+0000", 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9232000000000004e-05 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.078190+0000", 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8418000000000005e-05 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080706+0000", 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025159370000000002 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484248+0000", 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40354268999999998 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484325+0000", 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.661e-05 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484427+0000", 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102061 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.294 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074328+0000", 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.1398409960000002, 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41032785100000002, 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 137 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074328+0000", 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074328+0000", 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074329+0000", 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2999999999999999e-07 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074467+0000", 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013814200000000001 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074500+0000", 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3067e-05 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074518+0000", 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7591000000000001e-05 2024-04-18T23:10:54.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.078498+0000", 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039798180000000004 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.078553+0000", 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5544000000000002e-05 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.078644+0000", 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0730999999999996e-05 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081132+0000", 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024878919999999998 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484517+0000", 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40338511799999999 2024-04-18T23:10:54.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484589+0000", 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1637000000000005e-05 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484656+0000", 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7481e-05 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074542+0000", 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.1396277469999996, 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.410369231, 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.301 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 138 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074542+0000", 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074542+0000", 2024-04-18T23:10:54.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074543+0000", 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8190000000000001e-06 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074552+0000", 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8659999999999995e-06 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074568+0000", 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5622000000000001e-05 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074576+0000", 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6930000000000007e-06 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.079047+0000", 2024-04-18T23:10:54.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0044718359999999999 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.079103+0000", 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5142999999999997e-05 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.079189+0000", 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6599000000000002e-05 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081137+0000", 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001947444 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484736+0000", 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.403599237 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484811+0000", 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4900000000000005e-05 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484911+0000", 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100072 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.307 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074590+0000", 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.1395791119999998, 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41053327499999998, 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 139 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074590+0000", 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074590+0000", 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074591+0000", 2024-04-18T23:10:54.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7700000000000004e-07 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074596+0000", 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.147e-06 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074605+0000", 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9849999999999995e-06 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074612+0000", 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1430000000000001e-06 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.079528+0000", 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049156360000000001 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.079592+0000", 2024-04-18T23:10:54.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4171999999999995e-05 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.079701+0000", 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109412 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081140+0000", 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014384409999999999 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.484997+0000", 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.403856673 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485069+0000", 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2466999999999996e-05 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485124+0000", 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4521999999999997e-05 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.1395463770000003, 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.410819936, 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.314 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 140 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074624+0000", 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074804+0000", 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018036800000000001 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074824+0000", 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0018e-05 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074839+0000", 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4846e-05 2024-04-18T23:10:54.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080014+0000", 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051746780000000003 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080068+0000", 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4345000000000002e-05 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080162+0000", 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4157e-05 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081144+0000", 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098186100000000011 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485220+0000", 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40407592199999998 2024-04-18T23:10:54.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485314+0000", 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4161999999999997e-05 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485443+0000", 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012892900000000001 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.320 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.1393128140000002, 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41085222500000002, 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.321 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 141 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074858+0000", 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0139999999999999e-06 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074863+0000", 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3759999999999999e-06 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074872+0000", 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3929999999999997e-06 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074883+0000", 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0722e-05 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080730+0000", 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058473249999999996 2024-04-18T23:10:54.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080788+0000", 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7423999999999998e-05 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080881+0000", 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2938999999999999e-05 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081458+0000", 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057760199999999998 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485586+0000", 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40412787700000002 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485656+0000", 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0034000000000001e-05 2024-04-18T23:10:54.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485709+0000", 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2519000000000003e-05 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.327 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0800872699999999, 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50946670199999999, 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 162 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134083+0000", 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7000000000000003e-07 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134088+0000", 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2719999999999997e-06 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134097+0000", 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1400000000000006e-06 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134104+0000", 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140241+0000", 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0061371919999999996 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140319+0000", 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7768000000000005e-05 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140446+0000", 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126637 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243778+0000", 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103331863 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643391+0000", 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.39961375599999999 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643476+0000", 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4274e-05 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643549+0000", 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3071000000000006e-05 2024-04-18T23:10:54.333 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0800541099999998, 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50963481200000005, 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.334 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 163 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134116+0000", 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134123+0000", 2024-04-18T23:10:54.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4930000000000003e-06 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134131+0000", 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9490000000000002e-06 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134138+0000", 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8399999999999997e-06 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140848+0000", 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067093389999999999 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140969+0000", 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121787 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141164+0000", 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019482700000000001 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243782+0000", 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10261785900000001 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643626+0000", 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.399843858 2024-04-18T23:10:54.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643699+0000", 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2657999999999995e-05 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643750+0000", 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1481999999999999e-05 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.080020019, 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50988570200000005, 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.341 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 164 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134150+0000", 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4000000000000001e-07 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134332+0000", 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018177199999999999 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134351+0000", 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9321999999999999e-05 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134368+0000", 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6994999999999999e-05 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141740+0000", 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073716420000000003 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141823+0000", 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3042999999999998e-05 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141951+0000", 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127976 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243787+0000", 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101836063 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643874+0000", 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40008676199999998 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643964+0000", 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0362e-05 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.644035+0000", 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1124999999999994e-05 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0797607830000002, 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.54739116099999996, 2024-04-18T23:10:54.347 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 165 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:10:54.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134410+0000", 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5770000000000001e-06 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134420+0000", 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0175e-05 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134439+0000", 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8125e-05 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134453+0000", 2024-04-18T23:10:54.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4956e-05 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242648+0000", 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.108194819 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242783+0000", 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013431599999999999 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242992+0000", 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020903 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.244280+0000", 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012883669999999999 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681587+0000", 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.43730722300000002 2024-04-18T23:10:54.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681676+0000", 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8776000000000005e-05 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681800+0000", 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012379700000000001 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.353 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:10:54.354 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:10:54.354 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:10:54.355 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:10:54.355 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:10:54.356 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:10:54.356 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:54.357+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:10:54.412 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:10:54.412 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:10:54.484 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:54.482+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:10:54.485 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-04-18T23:10:54.485 INFO:teuthology.orchestra.run.smithi073.stderr:enabled application 'rados' on pool 'unique_pool_0' 2024-04-18T23:10:54.491 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:54.491 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:10:54.491 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:10:54.491 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:54.511 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:10:54.556 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:10:54.553+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:10:54.635 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:54.649 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:10:54.657 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:54.653+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:54.758 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:54.757+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:1 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'1, mlcod=0'0)", 2024-04-18T23:10:54.770 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:42.687401+0000", 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.084439679000001, 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198358914, 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687401+0000", 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687401+0000", 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687402+0000", 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687414+0000", 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2843000000000001e-05 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687440+0000", 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5398e-05 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687463+0000", 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2901e-05 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.882317+0000", 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19485413500000001 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.882396+0000", 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8681000000000001e-05 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.885671+0000", 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032758829999999998 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.885760+0000", 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8113000000000002e-05 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.774 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:114 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'76, mlcod=28'72)", 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.011152+0000", 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.7606885889999999, 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20343882299999999, 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011152+0000", 2024-04-18T23:10:54.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011152+0000", 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011154+0000", 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.013e-06 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011169+0000", 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5665999999999999e-05 2024-04-18T23:10:54.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011194+0000", 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4170999999999999e-05 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011209+0000", 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5766000000000001e-05 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.111484+0000", 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100274822 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.111517+0000", 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2700999999999997e-05 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214524+0000", 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103007425 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214591+0000", 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6259000000000004e-05 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.778 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:115 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'77, mlcod=28'72)", 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.014595+0000", 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.7572450479999997, 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20003684299999999, 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014595+0000", 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014595+0000", 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014596+0000", 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.153e-06 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014657+0000", 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0216000000000001e-05 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014676+0000", 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9562e-05 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014692+0000", 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5432999999999999e-05 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.113194+0000", 2024-04-18T23:10:54.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098502135000000005 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.113228+0000", 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3834999999999997e-05 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214596+0000", 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101368415 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214632+0000", 2024-04-18T23:10:54.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6093999999999997e-05 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:116 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'78, mlcod=28'74)", 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.016683+0000", 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.7551573649999996, 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19884875900000001, 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016683+0000", 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016683+0000", 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016685+0000", 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8330000000000001e-06 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016696+0000", 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1396999999999999e-05 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016725+0000", 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8977999999999999e-05 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016748+0000", 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2728e-05 2024-04-18T23:10:54.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214097+0000", 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19734953299999999 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214130+0000", 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2988000000000003e-05 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.215452+0000", 2024-04-18T23:10:54.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013212670000000001 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.215532+0000", 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0035000000000002e-05 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:117 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'79, mlcod=28'74)", 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.017827+0000", 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.7540136440000005, 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20024583600000001, 2024-04-18T23:10:54.787 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017827+0000", 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017827+0000", 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017829+0000", 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8199999999999999e-06 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017840+0000", 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1392e-05 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017868+0000", 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7789000000000001e-05 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017887+0000", 2024-04-18T23:10:54.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8868999999999998e-05 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214695+0000", 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19680872499999999 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214729+0000", 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4060999999999998e-05 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.217987+0000", 2024-04-18T23:10:54.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032573459999999999 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.218073+0000", 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5834000000000005e-05 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:123 1.0 e28/22 1:30306672:devicehealth::main.db.0000000000000000:head v 28'85, mlcod=28'83)", 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.728930+0000", 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0429107440000003, 2024-04-18T23:10:54.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201634012, 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728930+0000", 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728930+0000", 2024-04-18T23:10:54.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728931+0000", 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.316e-06 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728939+0000", 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3659999999999992e-06 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728967+0000", 2024-04-18T23:10:54.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7818999999999999e-05 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.829109+0000", 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100142192 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.929412+0000", 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10030241300000001 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.929448+0000", 2024-04-18T23:10:54.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6331000000000002e-05 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.930477+0000", 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001028927 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.930564+0000", 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6648000000000001e-05 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.795 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:133 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'91, mlcod=28'85)", 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.4944185839999999, 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20553276400000001, 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:10:54.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277424+0000", 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.21e-06 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277436+0000", 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1693e-05 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277464+0000", 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8608000000000001e-05 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377595+0000", 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100130684 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377730+0000", 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013455499999999999 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377781+0000", 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1406999999999998e-05 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482870+0000", 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105089222 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482955+0000", 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4384999999999999e-05 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:139 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'97, mlcod=28'85)", 2024-04-18T23:10:54.799 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.378021+0000", 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3938188480000004, 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105977117, 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378021+0000", 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378021+0000", 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378022+0000", 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0699999999999998e-07 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378026+0000", 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1529999999999999e-06 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378034+0000", 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0630000000000006e-06 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378043+0000", 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2790000000000004e-06 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.481515+0000", 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103471701 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.481545+0000", 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0312e-05 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.483913+0000", 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002367854 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.483999+0000", 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5447999999999999e-05 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:140 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'98, mlcod=28'85)", 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.378051+0000", 2024-04-18T23:10:54.803 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3937889060000002, 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106478701, 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378051+0000", 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378051+0000", 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378052+0000", 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1e-07 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378112+0000", 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0532999999999998e-05 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378122+0000", 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4860000000000003e-06 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378131+0000", 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8729999999999996e-06 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482069+0000", 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10393820300000001 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482104+0000", 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5067999999999997e-05 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.484455+0000", 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002351002 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.484530+0000", 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5226000000000001e-05 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:141 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'99, mlcod=28'85)", 2024-04-18T23:10:54.807 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.378148+0000", 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3936922660000004, 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107084213, 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378148+0000", 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378148+0000", 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378149+0000", 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8400000000000005e-07 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378153+0000", 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7029999999999997e-06 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378164+0000", 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0276e-05 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378170+0000", 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2659999999999998e-06 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.483345+0000", 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105174774 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.483380+0000", 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5298e-05 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.485174+0000", 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017937420000000001 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.485232+0000", 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8669999999999999e-05 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:148 1.0 e28/22 1:30306672:devicehealth::main.db.0000000000000000:head v 28'106, mlcod=28'104)", 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.825319+0000", 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9465210949999996, 2024-04-18T23:10:54.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202174931, 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825319+0000", 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825319+0000", 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825326+0000", 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2439999999999996e-06 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825372+0000", 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5383000000000002e-05 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825403+0000", 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0667999999999998e-05 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.925555+0000", 2024-04-18T23:10:54.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100152493 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.025857+0000", 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100301604 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.025902+0000", 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5794000000000002e-05 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.027403+0000", 2024-04-18T23:10:54.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015009769999999999 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.027494+0000", 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0767999999999996e-05 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:155 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'109, mlcod=28'105)", 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6368804739999998, 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20593233399999999, 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.815 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134962+0000", 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8560000000000001e-06 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135074+0000", 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112357 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135099+0000", 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4705000000000001e-05 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135115+0000", 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5855999999999999e-05 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135494+0000", 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037909800000000002 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135551+0000", 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7176999999999998e-05 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340852+0000", 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20530157900000001 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340892+0000", 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9706000000000002e-05 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:156 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'110, mlcod=28'105)", 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6360738470000005, 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.205157487, 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135768+0000", 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2130000000000001e-06 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135779+0000", 2024-04-18T23:10:54.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1749000000000001e-05 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135794+0000", 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453e-05 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135807+0000", 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2599000000000001e-05 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136859+0000", 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010521199999999999 2024-04-18T23:10:54.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136911+0000", 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2096999999999999e-05 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340896+0000", 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20398571500000001 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340924+0000", 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7540999999999999e-05 2024-04-18T23:10:54.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:157 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'111, mlcod=28'105)", 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.6353591549999997, 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20611499799999999, 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:10:54.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136483+0000", 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.576e-06 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136499+0000", 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6529e-05 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136520+0000", 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0571999999999999e-05 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136532+0000", 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.223e-05 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237736+0000", 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10120357200000001 2024-04-18T23:10:54.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237776+0000", 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0397e-05 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342561+0000", 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104784615 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342596+0000", 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5506999999999997e-05 2024-04-18T23:10:54.826 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:159 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'113, mlcod=28'105)", 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.237427+0000", 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5344136920000002, 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105570162, 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237427+0000", 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237427+0000", 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237427+0000", 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8299999999999997e-07 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237433+0000", 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9100000000000002e-06 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237443+0000", 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0149000000000001e-05 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237454+0000", 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0913e-05 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.339804+0000", 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102349696 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.339857+0000", 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.308e-05 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342953+0000", 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030955829999999998 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342997+0000", 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4248000000000001e-05 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.830 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:160 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'114, mlcod=28'105)", 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.237496+0000", 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5343443360000002, 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105561851, 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237496+0000", 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237496+0000", 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237497+0000", 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9299999999999998e-07 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237505+0000", 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4030000000000007e-06 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.832 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237518+0000", 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2626000000000001e-05 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237524+0000", 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8199999999999999e-06 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340549+0000", 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103024993 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340572+0000", 2024-04-18T23:10:54.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2458000000000001e-05 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.343000+0000", 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024281620000000002 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.343058+0000", 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7896000000000003e-05 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:161 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'115, mlcod=28'105)", 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.237535+0000", 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.534305399, 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106699368, 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237535+0000", 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237535+0000", 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237535+0000", 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3700000000000001e-07 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237638+0000", 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000102264 2024-04-18T23:10:54.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237650+0000", 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2765999999999999e-05 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237658+0000", 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2830000000000004e-06 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.341001+0000", 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103343485 2024-04-18T23:10:54.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.341049+0000", 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7867000000000001e-05 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.344199+0000", 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031503690000000001 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.344234+0000", 2024-04-18T23:10:54.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4996999999999997e-05 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:162 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'116, mlcod=28'105)", 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.237716+0000", 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5341247720000002, 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106546947, 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237716+0000", 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237716+0000", 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237716+0000", 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4300000000000003e-07 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237722+0000", 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4e-06 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237734+0000", 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2143e-05 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237741+0000", 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1590000000000004e-06 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342297+0000", 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10455627100000001 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342316+0000", 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9303000000000001e-05 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.344237+0000", 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001920876 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.344263+0000", 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5151999999999999e-05 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:163 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'117, mlcod=28'105)", 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.237753+0000", 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.5340877370000001, 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.106534562, 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237753+0000", 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237753+0000", 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237753+0000", 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.03e-07 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237764+0000", 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0786e-05 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237772+0000", 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2360000000000004e-06 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237778+0000", 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9030000000000001e-06 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342654+0000", 2024-04-18T23:10:54.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104876212 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342673+0000", 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8923000000000002e-05 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.344264+0000", 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001590773 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.344287+0000", 2024-04-18T23:10:54.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3425999999999999e-05 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:164 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'118, mlcod=28'105)", 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.237791+0000", 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.53404937, 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10769007799999999, 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237791+0000", 2024-04-18T23:10:54.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237791+0000", 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237791+0000", 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4700000000000002e-07 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237871+0000", 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9401999999999996e-05 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237880+0000", 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8459999999999997e-06 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237886+0000", 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2230000000000004e-06 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.343134+0000", 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105248278 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.343183+0000", 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8810999999999997e-05 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.345405+0000", 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022219010000000001 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.345481+0000", 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.627e-05 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.850 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:10:54.851 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:10:54.851 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:54.852 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:10:54.859 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:54.857+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:10:54.960 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:54.957+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:10:54.978 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:54.981 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:10:54.982 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:10:54.982 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:54.992 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:10:55.062 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:55.061+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:10:55.108 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:55.108 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:10:55.108 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:10:55.109 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:10:55.109 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:55.123 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:10:55.163 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:55.158+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:10:55.245 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:55.245 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:10:55.245 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:10:55.245 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2024-04-18T23:10:55.246 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:55.259 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:10:55.264 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:55.262+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:10:55.365 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:10:55.362+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:10:55.383 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:10:55.383 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:10:55.383 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:55.397 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:10:55.466 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:55.461+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:55.515 DEBUG:teuthology.orchestra.run.smithi142:> CEPH_CLIENT_ID=0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_rados --no-omap --ec-pool --max-ops 400 --objects 50 --max-in-flight 16 --size 4000000 --min-stride-size 400000 --max-stride-size 800000 --max-seconds 0 --op read 100 --op write 0 --op delete 50 --op snap_create 50 --op snap_remove 50 --op rollback 50 --op setattr 25 --op rmattr 25 --op copy_from 50 --op append 100 --pool unique_pool_0 2024-04-18T23:10:55.516 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:55.516 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:10:55.516 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:10:55.516 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:10:55.516 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:55.530 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:10:55.538 INFO:tasks.rados.rados.0.smithi142.stdout:adding op weight read -> 100 2024-04-18T23:10:55.538 INFO:tasks.rados.rados.0.smithi142.stdout:adding op weight delete -> 50 2024-04-18T23:10:55.538 INFO:tasks.rados.rados.0.smithi142.stdout:adding op weight snap_create -> 50 2024-04-18T23:10:55.538 INFO:tasks.rados.rados.0.smithi142.stdout:adding op weight snap_remove -> 50 2024-04-18T23:10:55.538 INFO:tasks.rados.rados.0.smithi142.stdout:adding op weight rollback -> 50 2024-04-18T23:10:55.538 INFO:tasks.rados.rados.0.smithi142.stdout:adding op weight setattr -> 25 2024-04-18T23:10:55.538 INFO:tasks.rados.rados.0.smithi142.stdout:adding op weight rmattr -> 25 2024-04-18T23:10:55.538 INFO:tasks.rados.rados.0.smithi142.stdout:adding op weight copy_from -> 50 2024-04-18T23:10:55.539 INFO:tasks.rados.rados.0.smithi142.stdout:adding op weight append -> 100 2024-04-18T23:10:55.539 INFO:tasks.rados.rados.0.smithi142.stdout:ceph version 19.0.0-2369-g1a6a3e74 (1a6a3e742fd81c6d70be70552499bff767652c32) squid (dev) 2024-04-18T23:10:55.539 INFO:tasks.rados.rados.0.smithi142.stdout:Configuration: 2024-04-18T23:10:55.539 INFO:tasks.rados.rados.0.smithi142.stdout: Number of operations: 400 2024-04-18T23:10:55.539 INFO:tasks.rados.rados.0.smithi142.stdout: Number of objects: 50 2024-04-18T23:10:55.539 INFO:tasks.rados.rados.0.smithi142.stdout: Max in flight operations: 16 2024-04-18T23:10:55.539 INFO:tasks.rados.rados.0.smithi142.stdout: Object size (in bytes): 4000000 2024-04-18T23:10:55.539 INFO:tasks.rados.rados.0.smithi142.stdout: Write stride min: 400000 2024-04-18T23:10:55.539 INFO:tasks.rados.rados.0.smithi142.stdout: Write stride max: 800000 2024-04-18T23:10:55.567 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:10:55.565+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:10:55.653 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:55.653 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:10:55.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:10:55.653 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2024-04-18T23:10:55.653 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:55.667 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:10:55.668 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:55.665+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:55.768 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:55.766+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:10:55.791 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:55.791 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:10:55.791 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:10:55.791 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:55.805 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:10:55.869 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:55.869+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:55.926 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:55.927 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:10:55.927 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:10:55.927 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:10:55.927 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:55.940 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:10:55.971 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:55.969+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:10:56.062 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:10:56.062 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:10:56.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:10:56.062 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [] 2024-04-18T23:10:56.062 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:10:56.072 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:56.071+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:10:56.076 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:10:56.173 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:56.173+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:10:56.207 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:10:56.207 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:10:56.207 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:10:56.207 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:10:56.220 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:10:56.274 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:56.273+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:56.339 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:10:56.339 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:10:56.340 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:10:56.340 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:10:56.340 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:10:56.353 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:10:56.376 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:56.375+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:10:56.440 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:10:56.477 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:56.473+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:56.511 INFO:tasks.rados.rados.0.smithi142.stdout:1: write initial oid 1 2024-04-18T23:10:56.511 INFO:tasks.rados.rados.0.smithi142.stdout:1: seq_num 0 ranges {0=32768} 2024-04-18T23:10:56.512 INFO:tasks.rados.rados.0.smithi142.stdout:1: writing smithi14230322-1 from 0 to 32768 tid 1 2024-04-18T23:10:56.512 INFO:tasks.rados.rados.0.smithi142.stdout:2: write initial oid 2 2024-04-18T23:10:56.512 INFO:tasks.rados.rados.0.smithi142.stdout:2: seq_num 1 ranges {0=32768} 2024-04-18T23:10:56.512 INFO:tasks.rados.rados.0.smithi142.stdout:2: writing smithi14230322-2 from 0 to 32768 tid 1 2024-04-18T23:10:56.512 INFO:tasks.rados.rados.0.smithi142.stdout:3: write initial oid 3 2024-04-18T23:10:56.512 INFO:tasks.rados.rados.0.smithi142.stdout:3: seq_num 2 ranges {0=49152} 2024-04-18T23:10:56.513 INFO:tasks.rados.rados.0.smithi142.stdout:3: writing smithi14230322-3 from 0 to 49152 tid 1 2024-04-18T23:10:56.513 INFO:tasks.rados.rados.0.smithi142.stdout:4: write initial oid 4 2024-04-18T23:10:56.513 INFO:tasks.rados.rados.0.smithi142.stdout:4: seq_num 3 ranges {0=65536} 2024-04-18T23:10:56.513 INFO:tasks.rados.rados.0.smithi142.stdout:4: writing smithi14230322-4 from 0 to 65536 tid 1 2024-04-18T23:10:56.513 INFO:tasks.rados.rados.0.smithi142.stdout:5: write initial oid 5 2024-04-18T23:10:56.513 INFO:tasks.rados.rados.0.smithi142.stdout:5: seq_num 4 ranges {0=81920} 2024-04-18T23:10:56.514 INFO:tasks.rados.rados.0.smithi142.stdout:5: writing smithi14230322-5 from 0 to 81920 tid 1 2024-04-18T23:10:56.514 INFO:tasks.rados.rados.0.smithi142.stdout:6: write initial oid 6 2024-04-18T23:10:56.514 INFO:tasks.rados.rados.0.smithi142.stdout:6: seq_num 5 ranges {0=98304} 2024-04-18T23:10:56.514 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T23:10:56.515 INFO:tasks.rados.rados.0.smithi142.stdout:6: writing smithi14230322-6 from 0 to 98304 tid 1 2024-04-18T23:10:56.515 INFO:tasks.rados.rados.0.smithi142.stdout:7: write initial oid 7 2024-04-18T23:10:56.515 INFO:tasks.rados.rados.0.smithi142.stdout:7: seq_num 6 ranges {0=114688} 2024-04-18T23:10:56.516 INFO:tasks.rados.rados.0.smithi142.stdout:7: writing smithi14230322-7 from 0 to 114688 tid 1 2024-04-18T23:10:56.516 INFO:tasks.rados.rados.0.smithi142.stdout:8: write initial oid 8 2024-04-18T23:10:56.516 INFO:tasks.rados.rados.0.smithi142.stdout:8: seq_num 7 ranges {0=131072} 2024-04-18T23:10:56.516 INFO:tasks.rados.rados.0.smithi142.stdout:8: writing smithi14230322-8 from 0 to 131072 tid 1 2024-04-18T23:10:56.516 INFO:tasks.rados.rados.0.smithi142.stdout:9: write initial oid 9 2024-04-18T23:10:56.517 INFO:tasks.rados.rados.0.smithi142.stdout:9: seq_num 8 ranges {0=147456} 2024-04-18T23:10:56.518 INFO:tasks.rados.rados.0.smithi142.stdout:9: writing smithi14230322-9 from 0 to 147456 tid 1 2024-04-18T23:10:56.518 INFO:tasks.rados.rados.0.smithi142.stdout:10: write initial oid 10 2024-04-18T23:10:56.518 INFO:tasks.rados.rados.0.smithi142.stdout:10: seq_num 9 ranges {0=163840} 2024-04-18T23:10:56.520 INFO:tasks.rados.rados.0.smithi142.stdout:10: writing smithi14230322-10 from 0 to 163840 tid 1 2024-04-18T23:10:56.520 INFO:tasks.rados.rados.0.smithi142.stdout:11: write initial oid 11 2024-04-18T23:10:56.520 INFO:tasks.rados.rados.0.smithi142.stdout:11: seq_num 10 ranges {0=180224} 2024-04-18T23:10:56.522 INFO:tasks.rados.rados.0.smithi142.stdout:11: writing smithi14230322-11 from 0 to 180224 tid 1 2024-04-18T23:10:56.522 INFO:tasks.rados.rados.0.smithi142.stdout:12: write initial oid 12 2024-04-18T23:10:56.522 INFO:tasks.rados.rados.0.smithi142.stdout:12: seq_num 11 ranges {0=196608} 2024-04-18T23:10:56.523 INFO:tasks.rados.rados.0.smithi142.stdout:12: writing smithi14230322-12 from 0 to 196608 tid 1 2024-04-18T23:10:56.523 INFO:tasks.rados.rados.0.smithi142.stdout:13: write initial oid 13 2024-04-18T23:10:56.524 INFO:tasks.rados.rados.0.smithi142.stdout:13: seq_num 12 ranges {0=212992} 2024-04-18T23:10:56.524 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:10:56.524 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:10:56.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:10:56.524 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [] 2024-04-18T23:10:56.524 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:10:56.526 INFO:tasks.rados.rados.0.smithi142.stdout:13: writing smithi14230322-13 from 0 to 212992 tid 1 2024-04-18T23:10:56.526 INFO:tasks.rados.rados.0.smithi142.stdout:14: write initial oid 14 2024-04-18T23:10:56.526 INFO:tasks.rados.rados.0.smithi142.stdout:14: seq_num 13 ranges {0=229376} 2024-04-18T23:10:56.528 INFO:tasks.rados.rados.0.smithi142.stdout:14: writing smithi14230322-14 from 0 to 229376 tid 1 2024-04-18T23:10:56.528 INFO:tasks.rados.rados.0.smithi142.stdout:15: write initial oid 15 2024-04-18T23:10:56.528 INFO:tasks.rados.rados.0.smithi142.stdout:15: seq_num 14 ranges {0=245760} 2024-04-18T23:10:56.531 INFO:tasks.rados.rados.0.smithi142.stdout:15: writing smithi14230322-15 from 0 to 245760 tid 1 2024-04-18T23:10:56.531 INFO:tasks.rados.rados.0.smithi142.stdout:16: write initial oid 16 2024-04-18T23:10:56.531 INFO:tasks.rados.rados.0.smithi142.stdout:16: seq_num 15 ranges {0=262144} 2024-04-18T23:10:56.533 INFO:tasks.rados.rados.0.smithi142.stdout:16: writing smithi14230322-16 from 0 to 262144 tid 1 2024-04-18T23:10:56.533 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:56.543 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:10:56.590 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:56.589+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:56.640 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:10:56.640 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:10:56.679 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:56.679+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:10:56.716 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:10:56.716 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:28 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~163840 in=163840b,stat out=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.523499+0000", 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.19447435499999999, 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19452987499999999, 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:10:56.717 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 28 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523499+0000", 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523499+0000", 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:10:56.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523501+0000", 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.125e-06 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523671+0000", 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000170767 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523688+0000", 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6565000000000001e-05 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523699+0000", 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0825999999999999e-05 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523855+0000", 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015598399999999999 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523862+0000", 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5429999999999996e-06 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532410+0000", 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0085475919999999997 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:10:56.721 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532697+0000", 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028755700000000003 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.533837+0000", 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001139999 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.536785+0000", 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029477489999999999 2024-04-18T23:10:56.722 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:29 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [setxattr _header (56) in=63b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.523729+0000", 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.194243531, 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.194462686, 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:10:56.723 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 29 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523729+0000", 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523729+0000", 2024-04-18T23:10:56.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523730+0000", 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5899999999999995e-07 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523734+0000", 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.211e-06 2024-04-18T23:10:56.725 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523746+0000", 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1398e-05 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523771+0000", 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4817e-05 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:10:56.726 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523929+0000", 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015801500000000001 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523934+0000", 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3240000000000002e-06 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.534690+0000", 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010756533 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.727 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.534992+0000", 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030151700000000001 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.536263+0000", 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012710499999999999 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:10:56.728 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.540150+0000", 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038867789999999999 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:30 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.523786+0000", 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.194186741, 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.194550422, 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:10:56.729 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "waiting for rw locks", 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 30 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523786+0000", 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:10:56.730 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523786+0000", 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523787+0000", 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8800000000000002e-07 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.731 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523790+0000", 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5030000000000002e-06 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523799+0000", 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6440000000000006e-06 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.623870+0000", 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10007102900000001 2024-04-18T23:10:56.732 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.623946+0000", 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5525999999999995e-05 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624049+0000", 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000103501 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:10:56.733 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 34/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.637567+0000", 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.080405629000000006, 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.080884220000000007, 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.734 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.637567+0000", 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.637567+0000", 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:10:56.735 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.637568+0000", 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3300000000000002e-07 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.637570+0000", 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4310000000000001e-06 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.637580+0000", 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6840000000000008e-06 2024-04-18T23:10:56.736 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.637585+0000", 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1139999999999997e-06 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 4 2024-04-18T23:10:56.737 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:10:56.738 INFO:tasks.rados.rados.0.smithi142.stdout:7: finishing write tid 1 to smithi14230322-7 2024-04-18T23:10:56.738 INFO:tasks.rados.rados.0.smithi142.stdout:6: finishing write tid 1 to smithi14230322-6 2024-04-18T23:10:56.738 INFO:tasks.rados.rados.0.smithi142.stdout:6: finishing write tid 2 to smithi14230322-6 2024-04-18T23:10:56.739 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:10:56.740 INFO:tasks.rados.rados.0.smithi142.stdout:3: finishing write tid 1 to smithi14230322-3 2024-04-18T23:10:56.740 INFO:tasks.rados.rados.0.smithi142.stdout:3: finishing write tid 2 to smithi14230322-3 2024-04-18T23:10:56.747 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:10:56.747 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:10:56.775 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:10:56.775 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:10:56.789 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:56.789+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:56.798 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:10:56.799 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:10:56.833 INFO:tasks.rados.rados.0.smithi142.stdout:1: finishing write tid 1 to smithi14230322-1 2024-04-18T23:10:56.833 INFO:tasks.rados.rados.0.smithi142.stdout:1: finishing write tid 2 to smithi14230322-1 2024-04-18T23:10:56.833 INFO:tasks.rados.rados.0.smithi142.stdout:7: finishing write tid 2 to smithi14230322-7 2024-04-18T23:10:56.833 INFO:tasks.rados.rados.0.smithi142.stdout:10: finishing write tid 1 to smithi14230322-10 2024-04-18T23:10:56.834 INFO:tasks.rados.rados.0.smithi142.stdout:10: finishing write tid 2 to smithi14230322-10 2024-04-18T23:10:56.836 INFO:tasks.rados.rados.0.smithi142.stdout:7: finishing write tid 3 to smithi14230322-7 2024-04-18T23:10:56.836 INFO:tasks.rados.rados.0.smithi142.stdout:7: oid 7 updating version 0 to 1 2024-04-18T23:10:56.836 INFO:tasks.rados.rados.0.smithi142.stdout:7: oid 7 updating version 1 to 2 2024-04-18T23:10:56.836 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 7 v 2 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-18T23:10:56.836 INFO:tasks.rados.rados.0.smithi142.stdout:7: left oid 7 (ObjNum 6 snap 0 seq_num 6) 2024-04-18T23:10:56.836 INFO:tasks.rados.rados.0.smithi142.stdout:7: done (15 left) 2024-04-18T23:10:56.836 INFO:tasks.rados.rados.0.smithi142.stdout:17: write initial oid 17 2024-04-18T23:10:56.837 INFO:tasks.rados.rados.0.smithi142.stdout:17: seq_num 16 ranges {0=278528} 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout:17: writing smithi14230322-17 from 0 to 278528 tid 1 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout:3: finishing write tid 3 to smithi14230322-3 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout:3: oid 3 updating version 0 to 1 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout:3: oid 3 updating version 1 to 2 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 3 v 2 (ObjNum 2 snap 0 seq_num 2) dirty exists 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout:3: left oid 3 (ObjNum 2 snap 0 seq_num 2) 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout:3: done (15 left) 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout:18: write initial oid 18 2024-04-18T23:10:56.838 INFO:tasks.rados.rados.0.smithi142.stdout:18: seq_num 17 ranges {0=294912} 2024-04-18T23:10:56.839 INFO:tasks.rados.rados.0.smithi142.stdout:18: writing smithi14230322-18 from 0 to 294912 tid 1 2024-04-18T23:10:56.839 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:56.861 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:10:56.862 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:10:56.864 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:10:56.864 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:10:56.864 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:10:56.864 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:10:56.864 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:10:56.878 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:10:56.881 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:56.877+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:56.892 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:10:56.893 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:10:56.924 INFO:tasks.rados.rados.0.smithi142.stdout:15: finishing write tid 1 to smithi14230322-15 2024-04-18T23:10:56.924 INFO:tasks.rados.rados.0.smithi142.stdout:15: finishing write tid 2 to smithi14230322-15 2024-04-18T23:10:56.928 INFO:tasks.rados.rados.0.smithi142.stdout:13: finishing write tid 1 to smithi14230322-13 2024-04-18T23:10:56.928 INFO:tasks.rados.rados.0.smithi142.stdout:13: finishing write tid 2 to smithi14230322-13 2024-04-18T23:10:56.931 INFO:tasks.rados.rados.0.smithi142.stdout:1: finishing write tid 3 to smithi14230322-1 2024-04-18T23:10:56.931 INFO:tasks.rados.rados.0.smithi142.stdout:1: oid 1 updating version 0 to 1 2024-04-18T23:10:56.932 INFO:tasks.rados.rados.0.smithi142.stdout:1: oid 1 updating version 1 to 2 2024-04-18T23:10:56.932 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 1 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-04-18T23:10:56.932 INFO:tasks.rados.rados.0.smithi142.stdout:1: left oid 1 (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:10:56.932 INFO:tasks.rados.rados.0.smithi142.stdout:1: done (15 left) 2024-04-18T23:10:56.932 INFO:tasks.rados.rados.0.smithi142.stdout:19: write initial oid 19 2024-04-18T23:10:56.932 INFO:tasks.rados.rados.0.smithi142.stdout:19: seq_num 18 ranges {0=311296} 2024-04-18T23:10:56.934 INFO:tasks.rados.rados.0.smithi142.stdout:19: writing smithi14230322-19 from 0 to 311296 tid 1 2024-04-18T23:10:56.934 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:56.935 INFO:tasks.rados.rados.0.smithi142.stdout:13: finishing write tid 3 to smithi14230322-13 2024-04-18T23:10:56.935 INFO:tasks.rados.rados.0.smithi142.stdout:13: oid 13 updating version 0 to 1 2024-04-18T23:10:56.936 INFO:tasks.rados.rados.0.smithi142.stdout:13: oid 13 updating version 1 to 2 2024-04-18T23:10:56.936 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 13 v 2 (ObjNum 12 snap 0 seq_num 12) dirty exists 2024-04-18T23:10:56.936 INFO:tasks.rados.rados.0.smithi142.stdout:13: left oid 13 (ObjNum 12 snap 0 seq_num 12) 2024-04-18T23:10:56.936 INFO:tasks.rados.rados.0.smithi142.stdout:13: done (15 left) 2024-04-18T23:10:56.936 INFO:tasks.rados.rados.0.smithi142.stdout:20: write initial oid 20 2024-04-18T23:10:56.936 INFO:tasks.rados.rados.0.smithi142.stdout:20: seq_num 19 ranges {0=327680} 2024-04-18T23:10:56.938 INFO:tasks.rados.rados.0.smithi142.stdout:20: writing smithi14230322-20 from 0 to 327680 tid 1 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout:12: finishing write tid 1 to smithi14230322-12 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout:6: finishing write tid 3 to smithi14230322-6 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout:6: oid 6 updating version 0 to 1 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout:6: oid 6 updating version 1 to 2 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 6 v 2 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout:6: left oid 6 (ObjNum 5 snap 0 seq_num 5) 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout:6: done (15 left) 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout:21: write initial oid 21 2024-04-18T23:10:56.939 INFO:tasks.rados.rados.0.smithi142.stdout:21: seq_num 20 ranges {0=344064} 2024-04-18T23:10:56.941 INFO:tasks.rados.rados.0.smithi142.stdout:21: writing smithi14230322-21 from 0 to 344064 tid 1 2024-04-18T23:10:56.942 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:56.968 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:10:56.968 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:10:56.981 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:10:56.978+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:10:56.998 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:10:56.998 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:10:57.027 INFO:tasks.rados.rados.0.smithi142.stdout:14: finishing write tid 1 to smithi14230322-14 2024-04-18T23:10:57.028 INFO:tasks.rados.rados.0.smithi142.stdout:4: finishing write tid 1 to smithi14230322-4 2024-04-18T23:10:57.028 INFO:tasks.rados.rados.0.smithi142.stdout:4: finishing write tid 2 to smithi14230322-4 2024-04-18T23:10:57.029 INFO:tasks.rados.rados.0.smithi142.stdout:15: finishing write tid 3 to smithi14230322-15 2024-04-18T23:10:57.029 INFO:tasks.rados.rados.0.smithi142.stdout:15: oid 15 updating version 0 to 1 2024-04-18T23:10:57.029 INFO:tasks.rados.rados.0.smithi142.stdout:15: oid 15 updating version 1 to 2 2024-04-18T23:10:57.029 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 15 v 2 (ObjNum 14 snap 0 seq_num 14) dirty exists 2024-04-18T23:10:57.029 INFO:tasks.rados.rados.0.smithi142.stdout:15: left oid 15 (ObjNum 14 snap 0 seq_num 14) 2024-04-18T23:10:57.029 INFO:tasks.rados.rados.0.smithi142.stdout:15: done (15 left) 2024-04-18T23:10:57.029 INFO:tasks.rados.rados.0.smithi142.stdout:22: write initial oid 22 2024-04-18T23:10:57.029 INFO:tasks.rados.rados.0.smithi142.stdout:22: seq_num 21 ranges {0=360448} 2024-04-18T23:10:57.031 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:10:57.031 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:10:57.035 INFO:tasks.rados.rados.0.smithi142.stdout:22: writing smithi14230322-22 from 0 to 360448 tid 1 2024-04-18T23:10:57.035 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.035 INFO:tasks.rados.rados.0.smithi142.stdout:2: finishing write tid 1 to smithi14230322-2 2024-04-18T23:10:57.035 INFO:tasks.rados.rados.0.smithi142.stdout:2: finishing write tid 2 to smithi14230322-2 2024-04-18T23:10:57.036 INFO:tasks.rados.rados.0.smithi142.stdout:14: finishing write tid 2 to smithi14230322-14 2024-04-18T23:10:57.036 INFO:tasks.rados.rados.0.smithi142.stdout:17: finishing write tid 1 to smithi14230322-17 2024-04-18T23:10:57.039 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:57.039 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:57.053 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:10:57.059 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:10:57.060 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:10:57.086 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:10:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:10:57.097 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:57.097+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:10:57.124 INFO:tasks.rados.rados.0.smithi142.stdout:5: finishing write tid 1 to smithi14230322-5 2024-04-18T23:10:57.124 INFO:tasks.rados.rados.0.smithi142.stdout:5: finishing write tid 2 to smithi14230322-5 2024-04-18T23:10:57.129 INFO:tasks.rados.rados.0.smithi142.stdout:9: finishing write tid 1 to smithi14230322-9 2024-04-18T23:10:57.129 INFO:tasks.rados.rados.0.smithi142.stdout:9: finishing write tid 2 to smithi14230322-9 2024-04-18T23:10:57.130 INFO:tasks.rados.rados.0.smithi142.stdout:11: finishing write tid 1 to smithi14230322-11 2024-04-18T23:10:57.132 INFO:tasks.rados.rados.0.smithi142.stdout:11: finishing write tid 2 to smithi14230322-11 2024-04-18T23:10:57.135 INFO:tasks.rados.rados.0.smithi142.stdout:8: finishing write tid 1 to smithi14230322-8 2024-04-18T23:10:57.137 INFO:tasks.rados.rados.0.smithi142.stdout:16: finishing write tid 1 to smithi14230322-16 2024-04-18T23:10:57.137 INFO:tasks.rados.rados.0.smithi142.stdout:16: finishing write tid 2 to smithi14230322-16 2024-04-18T23:10:57.138 INFO:tasks.rados.rados.0.smithi142.stdout:10: finishing write tid 3 to smithi14230322-10 2024-04-18T23:10:57.138 INFO:tasks.rados.rados.0.smithi142.stdout:10: oid 10 updating version 0 to 1 2024-04-18T23:10:57.138 INFO:tasks.rados.rados.0.smithi142.stdout:10: oid 10 updating version 1 to 2 2024-04-18T23:10:57.138 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 10 v 2 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-18T23:10:57.138 INFO:tasks.rados.rados.0.smithi142.stdout:10: left oid 10 (ObjNum 9 snap 0 seq_num 9) 2024-04-18T23:10:57.138 INFO:tasks.rados.rados.0.smithi142.stdout:10: done (15 left) 2024-04-18T23:10:57.138 INFO:tasks.rados.rados.0.smithi142.stdout:23: write initial oid 23 2024-04-18T23:10:57.138 INFO:tasks.rados.rados.0.smithi142.stdout:23: seq_num 22 ranges {0=376832} 2024-04-18T23:10:57.141 INFO:tasks.rados.rados.0.smithi142.stdout:23: writing smithi14230322-23 from 0 to 376832 tid 1 2024-04-18T23:10:57.141 INFO:tasks.rados.rados.0.smithi142.stdout:14: finishing write tid 3 to smithi14230322-14 2024-04-18T23:10:57.141 INFO:tasks.rados.rados.0.smithi142.stdout:14: oid 14 updating version 0 to 3 2024-04-18T23:10:57.141 INFO:tasks.rados.rados.0.smithi142.stdout:14: oid 14 updating version 3 to 4 2024-04-18T23:10:57.141 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 14 v 4 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-18T23:10:57.141 INFO:tasks.rados.rados.0.smithi142.stdout:14: left oid 14 (ObjNum 13 snap 0 seq_num 13) 2024-04-18T23:10:57.141 INFO:tasks.rados.rados.0.smithi142.stdout:14: done (15 left) 2024-04-18T23:10:57.141 INFO:tasks.rados.rados.0.smithi142.stdout:24: write initial oid 24 2024-04-18T23:10:57.142 INFO:tasks.rados.rados.0.smithi142.stdout:24: seq_num 23 ranges {0=393216} 2024-04-18T23:10:57.144 INFO:tasks.rados.rados.0.smithi142.stdout:24: writing smithi14230322-24 from 0 to 393216 tid 1 2024-04-18T23:10:57.144 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.173 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:10:57.173 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:10:57.182 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:57.181+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:57.205 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:10:57.206 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:10:57.224 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:57.225 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:57.236 INFO:tasks.rados.rados.0.smithi142.stdout:8: finishing write tid 2 to smithi14230322-8 2024-04-18T23:10:57.239 INFO:tasks.rados.rados.0.smithi142.stdout:4: finishing write tid 3 to smithi14230322-4 2024-04-18T23:10:57.239 INFO:tasks.rados.rados.0.smithi142.stdout:4: oid 4 updating version 0 to 1 2024-04-18T23:10:57.239 INFO:tasks.rados.rados.0.smithi142.stdout:4: oid 4 updating version 1 to 2 2024-04-18T23:10:57.239 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 4 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-18T23:10:57.239 INFO:tasks.rados.rados.0.smithi142.stdout:4: left oid 4 (ObjNum 3 snap 0 seq_num 3) 2024-04-18T23:10:57.239 INFO:tasks.rados.rados.0.smithi142.stdout:4: done (15 left) 2024-04-18T23:10:57.239 INFO:tasks.rados.rados.0.smithi142.stdout:25: write initial oid 25 2024-04-18T23:10:57.239 INFO:tasks.rados.rados.0.smithi142.stdout:25: seq_num 24 ranges {0=409600} 2024-04-18T23:10:57.240 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:10:57.241 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:10:57.241 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:10:57.247 INFO:tasks.rados.rados.0.smithi142.stdout:25: writing smithi14230322-25 from 0 to 409600 tid 1 2024-04-18T23:10:57.247 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.269 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:10:57.269 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:10:57.284 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:10:57.281+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:10:57.337 INFO:tasks.rados.rados.0.smithi142.stdout:12: finishing write tid 2 to smithi14230322-12 2024-04-18T23:10:57.338 INFO:tasks.rados.rados.0.smithi142.stdout:20: finishing write tid 1 to smithi14230322-20 2024-04-18T23:10:57.344 INFO:tasks.rados.rados.0.smithi142.stdout:9: finishing write tid 3 to smithi14230322-9 2024-04-18T23:10:57.344 INFO:tasks.rados.rados.0.smithi142.stdout:9: oid 9 updating version 0 to 3 2024-04-18T23:10:57.344 INFO:tasks.rados.rados.0.smithi142.stdout:9: oid 9 updating version 3 to 4 2024-04-18T23:10:57.344 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 9 v 4 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-04-18T23:10:57.345 INFO:tasks.rados.rados.0.smithi142.stdout:9: left oid 9 (ObjNum 8 snap 0 seq_num 8) 2024-04-18T23:10:57.345 INFO:tasks.rados.rados.0.smithi142.stdout:9: done (15 left) 2024-04-18T23:10:57.345 INFO:tasks.rados.rados.0.smithi142.stdout:26: write initial oid 26 2024-04-18T23:10:57.345 INFO:tasks.rados.rados.0.smithi142.stdout:26: seq_num 25 ranges {0=425984} 2024-04-18T23:10:57.347 INFO:tasks.rados.rados.0.smithi142.stdout:26: writing smithi14230322-26 from 0 to 425984 tid 1 2024-04-18T23:10:57.347 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.347 INFO:tasks.rados.rados.0.smithi142.stdout:17: finishing write tid 2 to smithi14230322-17 2024-04-18T23:10:57.348 INFO:tasks.rados.rados.0.smithi142.stdout:23: finishing write tid 1 to smithi14230322-23 2024-04-18T23:10:57.370 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:57.371 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:57.384 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:10:57.382+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:10:57.385 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:10:57.427 INFO:tasks.rados.rados.0.smithi142.stdout:5: finishing write tid 3 to smithi14230322-5 2024-04-18T23:10:57.427 INFO:tasks.rados.rados.0.smithi142.stdout:5: oid 5 updating version 0 to 2 2024-04-18T23:10:57.427 INFO:tasks.rados.rados.0.smithi142.stdout:5: oid 5 version 2 is already newer than 1 2024-04-18T23:10:57.427 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 5 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-18T23:10:57.427 INFO:tasks.rados.rados.0.smithi142.stdout:5: left oid 5 (ObjNum 4 snap 0 seq_num 4) 2024-04-18T23:10:57.427 INFO:tasks.rados.rados.0.smithi142.stdout:5: done (15 left) 2024-04-18T23:10:57.427 INFO:tasks.rados.rados.0.smithi142.stdout:27: write initial oid 27 2024-04-18T23:10:57.428 INFO:tasks.rados.rados.0.smithi142.stdout:27: seq_num 26 ranges {0=442368} 2024-04-18T23:10:57.431 INFO:tasks.rados.rados.0.smithi142.stdout:27: writing smithi14230322-27 from 0 to 442368 tid 1 2024-04-18T23:10:57.431 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.431 INFO:tasks.rados.rados.0.smithi142.stdout:11: finishing write tid 3 to smithi14230322-11 2024-04-18T23:10:57.431 INFO:tasks.rados.rados.0.smithi142.stdout:11: oid 11 updating version 0 to 5 2024-04-18T23:10:57.431 INFO:tasks.rados.rados.0.smithi142.stdout:11: oid 11 updating version 5 to 6 2024-04-18T23:10:57.431 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 11 v 6 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-18T23:10:57.431 INFO:tasks.rados.rados.0.smithi142.stdout:11: left oid 11 (ObjNum 10 snap 0 seq_num 10) 2024-04-18T23:10:57.431 INFO:tasks.rados.rados.0.smithi142.stdout:11: done (15 left) 2024-04-18T23:10:57.432 INFO:tasks.rados.rados.0.smithi142.stdout:28: write initial oid 28 2024-04-18T23:10:57.432 INFO:tasks.rados.rados.0.smithi142.stdout:28: seq_num 27 ranges {0=458752} 2024-04-18T23:10:57.435 INFO:tasks.rados.rados.0.smithi142.stdout:28: writing smithi14230322-28 from 0 to 458752 tid 1 2024-04-18T23:10:57.435 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.438 INFO:tasks.rados.rados.0.smithi142.stdout:25: finishing write tid 1 to smithi14230322-25 2024-04-18T23:10:57.439 INFO:tasks.rados.rados.0.smithi142.stdout:16: finishing write tid 3 to smithi14230322-16 2024-04-18T23:10:57.439 INFO:tasks.rados.rados.0.smithi142.stdout:16: oid 16 updating version 0 to 2 2024-04-18T23:10:57.439 INFO:tasks.rados.rados.0.smithi142.stdout:16: oid 16 version 2 is already newer than 1 2024-04-18T23:10:57.439 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 16 v 2 (ObjNum 15 snap 0 seq_num 15) dirty exists 2024-04-18T23:10:57.439 INFO:tasks.rados.rados.0.smithi142.stdout:16: left oid 16 (ObjNum 15 snap 0 seq_num 15) 2024-04-18T23:10:57.439 INFO:tasks.rados.rados.0.smithi142.stdout:16: done (15 left) 2024-04-18T23:10:57.439 INFO:tasks.rados.rados.0.smithi142.stdout:29: write initial oid 29 2024-04-18T23:10:57.440 INFO:tasks.rados.rados.0.smithi142.stdout:29: seq_num 28 ranges {0=475136} 2024-04-18T23:10:57.443 INFO:tasks.rados.rados.0.smithi142.stdout:29: writing smithi14230322-29 from 0 to 475136 tid 1 2024-04-18T23:10:57.443 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.443 INFO:tasks.rados.rados.0.smithi142.stdout:23: finishing write tid 2 to smithi14230322-23 2024-04-18T23:10:57.444 INFO:tasks.rados.rados.0.smithi142.stdout:22: finishing write tid 1 to smithi14230322-22 2024-04-18T23:10:57.444 INFO:tasks.rados.rados.0.smithi142.stdout:22: finishing write tid 2 to smithi14230322-22 2024-04-18T23:10:57.485 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:57.483+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:10:57.511 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:57.511 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:57.525 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:10:57.526 INFO:tasks.rados.rados.0.smithi142.stdout:12: finishing write tid 3 to smithi14230322-12 2024-04-18T23:10:57.526 INFO:tasks.rados.rados.0.smithi142.stdout:12: oid 12 updating version 0 to 1 2024-04-18T23:10:57.526 INFO:tasks.rados.rados.0.smithi142.stdout:12: oid 12 updating version 1 to 2 2024-04-18T23:10:57.526 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 12 v 2 (ObjNum 11 snap 0 seq_num 11) dirty exists 2024-04-18T23:10:57.527 INFO:tasks.rados.rados.0.smithi142.stdout:12: left oid 12 (ObjNum 11 snap 0 seq_num 11) 2024-04-18T23:10:57.527 INFO:tasks.rados.rados.0.smithi142.stdout:12: done (15 left) 2024-04-18T23:10:57.527 INFO:tasks.rados.rados.0.smithi142.stdout:30: write initial oid 30 2024-04-18T23:10:57.527 INFO:tasks.rados.rados.0.smithi142.stdout:30: seq_num 29 ranges {0=491520} 2024-04-18T23:10:57.529 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-04-18T23:10:57.530 INFO:tasks.rados.rados.0.smithi142.stdout:30: writing smithi14230322-30 from 0 to 491520 tid 1 2024-04-18T23:10:57.530 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.541 INFO:tasks.rados.rados.0.smithi142.stdout:2: finishing write tid 3 to smithi14230322-2 2024-04-18T23:10:57.541 INFO:tasks.rados.rados.0.smithi142.stdout:2: oid 2 updating version 0 to 2 2024-04-18T23:10:57.541 INFO:tasks.rados.rados.0.smithi142.stdout:2: oid 2 version 2 is already newer than 1 2024-04-18T23:10:57.542 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 2 v 2 (ObjNum 1 snap 0 seq_num 1) dirty exists 2024-04-18T23:10:57.542 INFO:tasks.rados.rados.0.smithi142.stdout:2: left oid 2 (ObjNum 1 snap 0 seq_num 1) 2024-04-18T23:10:57.542 INFO:tasks.rados.rados.0.smithi142.stdout:2: done (15 left) 2024-04-18T23:10:57.542 INFO:tasks.rados.rados.0.smithi142.stdout:31: write initial oid 31 2024-04-18T23:10:57.542 INFO:tasks.rados.rados.0.smithi142.stdout:31: seq_num 30 ranges {0=507904} 2024-04-18T23:10:57.547 INFO:tasks.rados.rados.0.smithi142.stdout:31: writing smithi14230322-31 from 0 to 507904 tid 1 2024-04-18T23:10:57.547 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.547 INFO:tasks.rados.rados.0.smithi142.stdout:8: finishing write tid 3 to smithi14230322-8 2024-04-18T23:10:57.547 INFO:tasks.rados.rados.0.smithi142.stdout:8: oid 8 updating version 0 to 4 2024-04-18T23:10:57.547 INFO:tasks.rados.rados.0.smithi142.stdout:8: oid 8 version 4 is already newer than 3 2024-04-18T23:10:57.547 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 8 v 4 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-18T23:10:57.547 INFO:tasks.rados.rados.0.smithi142.stdout:8: left oid 8 (ObjNum 7 snap 0 seq_num 7) 2024-04-18T23:10:57.548 INFO:tasks.rados.rados.0.smithi142.stdout:8: done (15 left) 2024-04-18T23:10:57.548 INFO:tasks.rados.rados.0.smithi142.stdout:32: write initial oid 32 2024-04-18T23:10:57.548 INFO:tasks.rados.rados.0.smithi142.stdout:32: seq_num 31 ranges {0=524288} 2024-04-18T23:10:57.552 INFO:tasks.rados.rados.0.smithi142.stdout:32: writing smithi14230322-32 from 0 to 524288 tid 1 2024-04-18T23:10:57.552 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.552 INFO:tasks.rados.rados.0.smithi142.stdout:25: finishing write tid 2 to smithi14230322-25 2024-04-18T23:10:57.552 INFO:tasks.rados.rados.0.smithi142.stdout:17: finishing write tid 3 to smithi14230322-17 2024-04-18T23:10:57.552 INFO:tasks.rados.rados.0.smithi142.stdout:17: oid 17 updating version 0 to 4 2024-04-18T23:10:57.552 INFO:tasks.rados.rados.0.smithi142.stdout:17: oid 17 version 4 is already newer than 3 2024-04-18T23:10:57.553 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 17 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-18T23:10:57.553 INFO:tasks.rados.rados.0.smithi142.stdout:17: left oid 17 (ObjNum 16 snap 0 seq_num 16) 2024-04-18T23:10:57.553 INFO:tasks.rados.rados.0.smithi142.stdout:19: finishing write tid 1 to smithi14230322-19 2024-04-18T23:10:57.553 INFO:tasks.rados.rados.0.smithi142.stdout:17: done (15 left) 2024-04-18T23:10:57.553 INFO:tasks.rados.rados.0.smithi142.stdout:33: write initial oid 33 2024-04-18T23:10:57.553 INFO:tasks.rados.rados.0.smithi142.stdout:33: seq_num 32 ranges {0=540672} 2024-04-18T23:10:57.557 INFO:tasks.rados.rados.0.smithi142.stdout:33: writing smithi14230322-33 from 0 to 540672 tid 1 2024-04-18T23:10:57.557 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.557 INFO:tasks.rados.rados.0.smithi142.stdout:19: finishing write tid 2 to smithi14230322-19 2024-04-18T23:10:57.559 INFO:tasks.rados.rados.0.smithi142.stdout:22: finishing write tid 3 to smithi14230322-22 2024-04-18T23:10:57.559 INFO:tasks.rados.rados.0.smithi142.stdout:22: oid 22 updating version 0 to 3 2024-04-18T23:10:57.559 INFO:tasks.rados.rados.0.smithi142.stdout:22: oid 22 updating version 3 to 4 2024-04-18T23:10:57.559 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 22 v 4 (ObjNum 21 snap 0 seq_num 21) dirty exists 2024-04-18T23:10:57.559 INFO:tasks.rados.rados.0.smithi142.stdout:22: left oid 22 (ObjNum 21 snap 0 seq_num 21) 2024-04-18T23:10:57.559 INFO:tasks.rados.rados.0.smithi142.stdout:22: done (15 left) 2024-04-18T23:10:57.559 INFO:tasks.rados.rados.0.smithi142.stdout:34: write initial oid 34 2024-04-18T23:10:57.559 INFO:tasks.rados.rados.0.smithi142.stdout:34: seq_num 33 ranges {0=557056} 2024-04-18T23:10:57.564 INFO:tasks.rados.rados.0.smithi142.stdout:34: writing smithi14230322-34 from 0 to 557056 tid 1 2024-04-18T23:10:57.564 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.585 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:57.585+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:10:57.634 INFO:tasks.rados.rados.0.smithi142.stdout:23: finishing write tid 3 to smithi14230322-23 2024-04-18T23:10:57.634 INFO:tasks.rados.rados.0.smithi142.stdout:23: oid 23 updating version 0 to 4 2024-04-18T23:10:57.634 INFO:tasks.rados.rados.0.smithi142.stdout:23: oid 23 version 4 is already newer than 3 2024-04-18T23:10:57.634 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 23 v 4 (ObjNum 22 snap 0 seq_num 22) dirty exists 2024-04-18T23:10:57.634 INFO:tasks.rados.rados.0.smithi142.stdout:23: left oid 23 (ObjNum 22 snap 0 seq_num 22) 2024-04-18T23:10:57.634 INFO:tasks.rados.rados.0.smithi142.stdout:23: done (15 left) 2024-04-18T23:10:57.634 INFO:tasks.rados.rados.0.smithi142.stdout:35: write initial oid 35 2024-04-18T23:10:57.635 INFO:tasks.rados.rados.0.smithi142.stdout:35: seq_num 34 ranges {0=573440} 2024-04-18T23:10:57.641 INFO:tasks.rados.rados.0.smithi142.stdout:35: writing smithi14230322-35 from 0 to 573440 tid 1 2024-04-18T23:10:57.641 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.644 INFO:tasks.rados.rados.0.smithi142.stdout:28: finishing write tid 1 to smithi14230322-28 2024-04-18T23:10:57.644 INFO:tasks.rados.rados.0.smithi142.stdout:28: finishing write tid 2 to smithi14230322-28 2024-04-18T23:10:57.645 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:10:57.645 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:10:57.650 INFO:tasks.rados.rados.0.smithi142.stdout:20: finishing write tid 2 to smithi14230322-20 2024-04-18T23:10:57.655 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:57.655 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:57.670 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-18T23:10:57.686 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:57.682+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:10:57.741 INFO:tasks.rados.rados.0.smithi142.stdout:21: finishing write tid 1 to smithi14230322-21 2024-04-18T23:10:57.741 INFO:tasks.rados.rados.0.smithi142.stdout:21: finishing write tid 2 to smithi14230322-21 2024-04-18T23:10:57.746 INFO:tasks.rados.rados.0.smithi142.stdout:31: finishing write tid 1 to smithi14230322-31 2024-04-18T23:10:57.746 INFO:tasks.rados.rados.0.smithi142.stdout:29: finishing write tid 1 to smithi14230322-29 2024-04-18T23:10:57.748 INFO:tasks.rados.rados.0.smithi142.stdout:25: finishing write tid 3 to smithi14230322-25 2024-04-18T23:10:57.748 INFO:tasks.rados.rados.0.smithi142.stdout:25: oid 25 updating version 0 to 2 2024-04-18T23:10:57.748 INFO:tasks.rados.rados.0.smithi142.stdout:25: oid 25 version 2 is already newer than 1 2024-04-18T23:10:57.748 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 25 v 2 (ObjNum 24 snap 0 seq_num 24) dirty exists 2024-04-18T23:10:57.748 INFO:tasks.rados.rados.0.smithi142.stdout:25: left oid 25 (ObjNum 24 snap 0 seq_num 24) 2024-04-18T23:10:57.748 INFO:tasks.rados.rados.0.smithi142.stdout:25: done (15 left) 2024-04-18T23:10:57.748 INFO:tasks.rados.rados.0.smithi142.stdout:36: write initial oid 36 2024-04-18T23:10:57.748 INFO:tasks.rados.rados.0.smithi142.stdout:36: seq_num 35 ranges {0=589824} 2024-04-18T23:10:57.752 INFO:tasks.rados.rados.0.smithi142.stdout:36: writing smithi14230322-36 from 0 to 589824 tid 1 2024-04-18T23:10:57.753 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.753 INFO:tasks.rados.rados.0.smithi142.stdout:18: finishing write tid 1 to smithi14230322-18 2024-04-18T23:10:57.754 INFO:tasks.rados.rados.0.smithi142.stdout:32: finishing write tid 1 to smithi14230322-32 2024-04-18T23:10:57.754 INFO:tasks.rados.rados.0.smithi142.stdout:31: finishing write tid 2 to smithi14230322-31 2024-04-18T23:10:57.754 INFO:tasks.rados.rados.0.smithi142.stdout:29: finishing write tid 2 to smithi14230322-29 2024-04-18T23:10:57.756 INFO:tasks.rados.rados.0.smithi142.stdout:19: finishing write tid 3 to smithi14230322-19 2024-04-18T23:10:57.757 INFO:tasks.rados.rados.0.smithi142.stdout:19: oid 19 updating version 0 to 8 2024-04-18T23:10:57.757 INFO:tasks.rados.rados.0.smithi142.stdout:19: oid 19 version 8 is already newer than 7 2024-04-18T23:10:57.757 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 19 v 8 (ObjNum 18 snap 0 seq_num 18) dirty exists 2024-04-18T23:10:57.757 INFO:tasks.rados.rados.0.smithi142.stdout:19: left oid 19 (ObjNum 18 snap 0 seq_num 18) 2024-04-18T23:10:57.757 INFO:tasks.rados.rados.0.smithi142.stdout:19: done (15 left) 2024-04-18T23:10:57.757 INFO:tasks.rados.rados.0.smithi142.stdout:37: write initial oid 37 2024-04-18T23:10:57.757 INFO:tasks.rados.rados.0.smithi142.stdout:37: seq_num 36 ranges {0=606208} 2024-04-18T23:10:57.761 INFO:tasks.rados.rados.0.smithi142.stdout:37: writing smithi14230322-37 from 0 to 606208 tid 1 2024-04-18T23:10:57.761 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.787 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:10:57.782+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:10:57.832 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:57.832 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:57.850 INFO:tasks.rados.rados.0.smithi142.stdout:28: finishing write tid 3 to smithi14230322-28 2024-04-18T23:10:57.851 INFO:tasks.rados.rados.0.smithi142.stdout:28: oid 28 updating version 0 to 3 2024-04-18T23:10:57.851 INFO:tasks.rados.rados.0.smithi142.stdout:28: oid 28 updating version 3 to 4 2024-04-18T23:10:57.851 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 28 v 4 (ObjNum 27 snap 0 seq_num 27) dirty exists 2024-04-18T23:10:57.851 INFO:tasks.rados.rados.0.smithi142.stdout:28: left oid 28 (ObjNum 27 snap 0 seq_num 27) 2024-04-18T23:10:57.851 INFO:tasks.rados.rados.0.smithi142.stdout:28: done (15 left) 2024-04-18T23:10:57.851 INFO:tasks.rados.rados.0.smithi142.stdout:38: write initial oid 38 2024-04-18T23:10:57.851 INFO:tasks.rados.rados.0.smithi142.stdout:38: seq_num 37 ranges {0=622592} 2024-04-18T23:10:57.852 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-18T23:10:57.859 INFO:tasks.rados.rados.0.smithi142.stdout:38: writing smithi14230322-38 from 0 to 622592 tid 1 2024-04-18T23:10:57.859 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.859 INFO:tasks.rados.rados.0.smithi142.stdout:18: finishing write tid 2 to smithi14230322-18 2024-04-18T23:10:57.859 INFO:tasks.rados.rados.0.smithi142.stdout:24: finishing write tid 1 to smithi14230322-24 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:24: finishing write tid 2 to smithi14230322-24 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:20: finishing write tid 3 to smithi14230322-20 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:20: oid 20 updating version 0 to 2 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:20: oid 20 version 2 is already newer than 1 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 20 v 2 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:20: left oid 20 (ObjNum 19 snap 0 seq_num 19) 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:32: finishing write tid 2 to smithi14230322-32 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:35: finishing write tid 1 to smithi14230322-35 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:20: done (15 left) 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:39: write initial oid 39 2024-04-18T23:10:57.860 INFO:tasks.rados.rados.0.smithi142.stdout:39: seq_num 38 ranges {0=638976} 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:39: writing smithi14230322-39 from 0 to 638976 tid 1 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:31: finishing write tid 3 to smithi14230322-31 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:31: oid 31 updating version 0 to 4 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:31: oid 31 version 4 is already newer than 3 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 31 v 4 (ObjNum 30 snap 0 seq_num 30) dirty exists 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:31: left oid 31 (ObjNum 30 snap 0 seq_num 30) 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:26: finishing write tid 1 to smithi14230322-26 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:26: finishing write tid 2 to smithi14230322-26 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:27: finishing write tid 1 to smithi14230322-27 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:31: done (15 left) 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:40: write initial oid 40 2024-04-18T23:10:57.871 INFO:tasks.rados.rados.0.smithi142.stdout:40: seq_num 39 ranges {0=671744} 2024-04-18T23:10:57.882 INFO:tasks.rados.rados.0.smithi142.stdout:40: writing smithi14230322-40 from 0 to 671744 tid 1 2024-04-18T23:10:57.882 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.882 INFO:tasks.rados.rados.0.smithi142.stdout:27: finishing write tid 2 to smithi14230322-27 2024-04-18T23:10:57.887 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:57.885+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:10:57.924 INFO:tasks.rados.rados.0.smithi142.stdout:30: finishing write tid 1 to smithi14230322-30 2024-04-18T23:10:57.925 INFO:tasks.rados.rados.0.smithi142.stdout:33: finishing write tid 1 to smithi14230322-33 2024-04-18T23:10:57.940 INFO:tasks.rados.rados.0.smithi142.stdout:34: finishing write tid 1 to smithi14230322-34 2024-04-18T23:10:57.940 INFO:tasks.rados.rados.0.smithi142.stdout:34: finishing write tid 2 to smithi14230322-34 2024-04-18T23:10:57.943 INFO:tasks.rados.rados.0.smithi142.stdout:35: finishing write tid 2 to smithi14230322-35 2024-04-18T23:10:57.945 INFO:tasks.rados.rados.0.smithi142.stdout:33: finishing write tid 2 to smithi14230322-33 2024-04-18T23:10:57.947 INFO:tasks.rados.rados.0.smithi142.stdout:29: finishing write tid 3 to smithi14230322-29 2024-04-18T23:10:57.948 INFO:tasks.rados.rados.0.smithi142.stdout:29: oid 29 updating version 0 to 4 2024-04-18T23:10:57.948 INFO:tasks.rados.rados.0.smithi142.stdout:29: oid 29 version 4 is already newer than 3 2024-04-18T23:10:57.948 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 29 v 4 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-18T23:10:57.948 INFO:tasks.rados.rados.0.smithi142.stdout:29: left oid 29 (ObjNum 28 snap 0 seq_num 28) 2024-04-18T23:10:57.948 INFO:tasks.rados.rados.0.smithi142.stdout:29: done (15 left) 2024-04-18T23:10:57.948 INFO:tasks.rados.rados.0.smithi142.stdout:41: write initial oid 41 2024-04-18T23:10:57.948 INFO:tasks.rados.rados.0.smithi142.stdout:41: seq_num 40 ranges {0=688128} 2024-04-18T23:10:57.953 INFO:tasks.rados.rados.0.smithi142.stdout:41: writing smithi14230322-41 from 0 to 688128 tid 1 2024-04-18T23:10:57.953 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:57.954 INFO:tasks.rados.rados.0.smithi142.stdout:30: finishing write tid 2 to smithi14230322-30 2024-04-18T23:10:57.980 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:57.980 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:57.988 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:57.987+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:10:57.994 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:10:58.056 INFO:tasks.rados.rados.0.smithi142.stdout:18: finishing write tid 3 to smithi14230322-18 2024-04-18T23:10:58.057 INFO:tasks.rados.rados.0.smithi142.stdout:18: oid 18 updating version 0 to 6 2024-04-18T23:10:58.057 INFO:tasks.rados.rados.0.smithi142.stdout:18: oid 18 version 6 is already newer than 5 2024-04-18T23:10:58.057 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 18 v 6 (ObjNum 17 snap 0 seq_num 17) dirty exists 2024-04-18T23:10:58.057 INFO:tasks.rados.rados.0.smithi142.stdout:18: left oid 18 (ObjNum 17 snap 0 seq_num 17) 2024-04-18T23:10:58.057 INFO:tasks.rados.rados.0.smithi142.stdout:18: done (15 left) 2024-04-18T23:10:58.057 INFO:tasks.rados.rados.0.smithi142.stdout:42: write initial oid 42 2024-04-18T23:10:58.057 INFO:tasks.rados.rados.0.smithi142.stdout:42: seq_num 41 ranges {0=704512} 2024-04-18T23:10:58.062 INFO:tasks.rados.rados.0.smithi142.stdout:42: writing smithi14230322-42 from 0 to 704512 tid 1 2024-04-18T23:10:58.062 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.062 INFO:tasks.rados.rados.0.smithi142.stdout:21: finishing write tid 3 to smithi14230322-21 2024-04-18T23:10:58.062 INFO:tasks.rados.rados.0.smithi142.stdout:21: oid 21 updating version 0 to 3 2024-04-18T23:10:58.062 INFO:tasks.rados.rados.0.smithi142.stdout:21: oid 21 updating version 3 to 4 2024-04-18T23:10:58.063 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 21 v 4 (ObjNum 20 snap 0 seq_num 20) dirty exists 2024-04-18T23:10:58.063 INFO:tasks.rados.rados.0.smithi142.stdout:21: left oid 21 (ObjNum 20 snap 0 seq_num 20) 2024-04-18T23:10:58.063 INFO:tasks.rados.rados.0.smithi142.stdout:21: done (15 left) 2024-04-18T23:10:58.063 INFO:tasks.rados.rados.0.smithi142.stdout:43: write initial oid 43 2024-04-18T23:10:58.063 INFO:tasks.rados.rados.0.smithi142.stdout:43: seq_num 42 ranges {0=720896} 2024-04-18T23:10:58.074 INFO:tasks.rados.rados.0.smithi142.stdout:43: writing smithi14230322-43 from 0 to 720896 tid 1 2024-04-18T23:10:58.074 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.074 INFO:tasks.rados.rados.0.smithi142.stdout:26: finishing write tid 3 to smithi14230322-26 2024-04-18T23:10:58.074 INFO:tasks.rados.rados.0.smithi142.stdout:26: oid 26 updating version 0 to 6 2024-04-18T23:10:58.074 INFO:tasks.rados.rados.0.smithi142.stdout:26: oid 26 version 6 is already newer than 5 2024-04-18T23:10:58.074 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 26 v 6 (ObjNum 25 snap 0 seq_num 25) dirty exists 2024-04-18T23:10:58.074 INFO:tasks.rados.rados.0.smithi142.stdout:26: left oid 26 (ObjNum 25 snap 0 seq_num 25) 2024-04-18T23:10:58.074 INFO:tasks.rados.rados.0.smithi142.stdout:26: done (15 left) 2024-04-18T23:10:58.074 INFO:tasks.rados.rados.0.smithi142.stdout:44: write initial oid 44 2024-04-18T23:10:58.075 INFO:tasks.rados.rados.0.smithi142.stdout:44: seq_num 43 ranges {0=737280} 2024-04-18T23:10:58.086 INFO:tasks.rados.rados.0.smithi142.stdout:44: writing smithi14230322-44 from 0 to 737280 tid 1 2024-04-18T23:10:58.086 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.086 INFO:tasks.rados.rados.0.smithi142.stdout:39: finishing write tid 1 to smithi14230322-39 2024-04-18T23:10:58.089 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:10:58.085+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:10:58.131 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:58.131 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:58.143 INFO:tasks.rados.rados.0.smithi142.stdout:34: finishing write tid 3 to smithi14230322-34 2024-04-18T23:10:58.143 INFO:tasks.rados.rados.0.smithi142.stdout:34: oid 34 updating version 0 to 3 2024-04-18T23:10:58.143 INFO:tasks.rados.rados.0.smithi142.stdout:34: oid 34 updating version 3 to 4 2024-04-18T23:10:58.143 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 34 v 4 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-04-18T23:10:58.144 INFO:tasks.rados.rados.0.smithi142.stdout:34: left oid 34 (ObjNum 33 snap 0 seq_num 33) 2024-04-18T23:10:58.144 INFO:tasks.rados.rados.0.smithi142.stdout:34: done (15 left) 2024-04-18T23:10:58.144 INFO:tasks.rados.rados.0.smithi142.stdout:45: write initial oid 45 2024-04-18T23:10:58.144 INFO:tasks.rados.rados.0.smithi142.stdout:45: seq_num 44 ranges {0=753664} 2024-04-18T23:10:58.145 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:10:58.155 INFO:tasks.rados.rados.0.smithi142.stdout:45: writing smithi14230322-45 from 0 to 753664 tid 1 2024-04-18T23:10:58.155 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.162 INFO:tasks.rados.rados.0.smithi142.stdout:30: finishing write tid 3 to smithi14230322-30 2024-04-18T23:10:58.163 INFO:tasks.rados.rados.0.smithi142.stdout:30: oid 30 updating version 0 to 4 2024-04-18T23:10:58.163 INFO:tasks.rados.rados.0.smithi142.stdout:30: oid 30 version 4 is already newer than 3 2024-04-18T23:10:58.163 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 30 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-18T23:10:58.163 INFO:tasks.rados.rados.0.smithi142.stdout:30: left oid 30 (ObjNum 29 snap 0 seq_num 29) 2024-04-18T23:10:58.163 INFO:tasks.rados.rados.0.smithi142.stdout:30: done (15 left) 2024-04-18T23:10:58.163 INFO:tasks.rados.rados.0.smithi142.stdout:46: write initial oid 46 2024-04-18T23:10:58.163 INFO:tasks.rados.rados.0.smithi142.stdout:46: seq_num 45 ranges {0=770048} 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:46: writing smithi14230322-46 from 0 to 770048 tid 1 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:27: finishing write tid 3 to smithi14230322-27 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:27: oid 27 updating version 0 to 8 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:27: oid 27 version 8 is already newer than 7 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 27 v 8 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:27: left oid 27 (ObjNum 26 snap 0 seq_num 26) 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:40: finishing write tid 1 to smithi14230322-40 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:27: done (15 left) 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:47: write initial oid 47 2024-04-18T23:10:58.177 INFO:tasks.rados.rados.0.smithi142.stdout:47: seq_num 46 ranges {0=786432} 2024-04-18T23:10:58.190 INFO:tasks.rados.rados.0.smithi142.stdout:47: writing smithi14230322-47 from 0 to 786432 tid 1 2024-04-18T23:10:58.190 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.190 INFO:tasks.rados.rados.0.smithi142.stdout:39: finishing write tid 2 to smithi14230322-39 2024-04-18T23:10:58.190 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:10:58.187+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:10:58.249 INFO:tasks.rados.rados.0.smithi142.stdout:33: finishing write tid 3 to smithi14230322-33 2024-04-18T23:10:58.249 INFO:tasks.rados.rados.0.smithi142.stdout:33: oid 33 updating version 0 to 4 2024-04-18T23:10:58.250 INFO:tasks.rados.rados.0.smithi142.stdout:33: oid 33 version 4 is already newer than 3 2024-04-18T23:10:58.250 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 33 v 4 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-18T23:10:58.250 INFO:tasks.rados.rados.0.smithi142.stdout:33: left oid 33 (ObjNum 32 snap 0 seq_num 32) 2024-04-18T23:10:58.250 INFO:tasks.rados.rados.0.smithi142.stdout:33: done (15 left) 2024-04-18T23:10:58.250 INFO:tasks.rados.rados.0.smithi142.stdout:48: write initial oid 48 2024-04-18T23:10:58.250 INFO:tasks.rados.rados.0.smithi142.stdout:48: seq_num 47 ranges {0=425984,425984=376832} 2024-04-18T23:10:58.257 INFO:tasks.rados.rados.0.smithi142.stdout:48: writing smithi14230322-48 from 0 to 425984 tid 1 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout:48: writing smithi14230322-48 from 425984 to 802816 tid 2 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout:24: finishing write tid 3 to smithi14230322-24 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout:24: oid 24 updating version 0 to 8 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout:24: oid 24 version 8 is already newer than 7 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 24 v 8 (ObjNum 23 snap 0 seq_num 23) dirty exists 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout:24: left oid 24 (ObjNum 23 snap 0 seq_num 23) 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout:24: done (15 left) 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout:49: write initial oid 49 2024-04-18T23:10:58.264 INFO:tasks.rados.rados.0.smithi142.stdout:49: seq_num 48 ranges {0=442368,442368=376832} 2024-04-18T23:10:58.268 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:58.268 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:58.271 INFO:tasks.rados.rados.0.smithi142.stdout:49: writing smithi14230322-49 from 0 to 442368 tid 1 2024-04-18T23:10:58.279 INFO:tasks.rados.rados.0.smithi142.stdout:49: writing smithi14230322-49 from 442368 to 819200 tid 2 2024-04-18T23:10:58.279 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.283 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:10:58.283 INFO:tasks.rados.rados.0.smithi142.stdout:43: finishing write tid 1 to smithi14230322-43 2024-04-18T23:10:58.284 INFO:tasks.rados.rados.0.smithi142.stdout:43: finishing write tid 2 to smithi14230322-43 2024-04-18T23:10:58.300 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-18T23:10:58.301 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-18T23:10:58.301 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-04-18T23:10:58.301 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-04-18T23:10:58.328 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:10:58.326+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:10:58.363 INFO:tasks.rados.rados.0.smithi142.stdout:41: finishing write tid 1 to smithi14230322-41 2024-04-18T23:10:58.392 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:10:58.391+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:10:58.406 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:58.406 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:58.421 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:35: finishing write tid 3 to smithi14230322-35 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:35: oid 35 updating version 0 to 5 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:35: oid 35 updating version 5 to 6 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 35 v 6 (ObjNum 34 snap 0 seq_num 34) dirty exists 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:35: left oid 35 (ObjNum 34 snap 0 seq_num 34) 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:36: finishing write tid 1 to smithi14230322-36 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:36: finishing write tid 2 to smithi14230322-36 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:35: done (15 left) 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:50: write initial oid 50 2024-04-18T23:10:58.460 INFO:tasks.rados.rados.0.smithi142.stdout:50: seq_num 49 ranges {0=458752,458752=376832} 2024-04-18T23:10:58.469 INFO:tasks.rados.rados.0.smithi142.stdout:50: writing smithi14230322-50 from 0 to 458752 tid 1 2024-04-18T23:10:58.478 INFO:tasks.rados.rados.0.smithi142.stdout:50: writing smithi14230322-50 from 458752 to 835584 tid 2 2024-04-18T23:10:58.478 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.478 INFO:tasks.rados.rados.0.smithi142.stdout:37: finishing write tid 1 to smithi14230322-37 2024-04-18T23:10:58.478 INFO:tasks.rados.rados.0.smithi142.stdout:37: finishing write tid 2 to smithi14230322-37 2024-04-18T23:10:58.478 INFO:tasks.rados.rados.0.smithi142.stdout:46: finishing write tid 1 to smithi14230322-46 2024-04-18T23:10:58.483 INFO:tasks.rados.rados.0.smithi142.stdout:39: finishing write tid 3 to smithi14230322-39 2024-04-18T23:10:58.483 INFO:tasks.rados.rados.0.smithi142.stdout:39: oid 39 updating version 0 to 5 2024-04-18T23:10:58.483 INFO:tasks.rados.rados.0.smithi142.stdout:39: oid 39 updating version 5 to 6 2024-04-18T23:10:58.483 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 39 v 6 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-18T23:10:58.483 INFO:tasks.rados.rados.0.smithi142.stdout:39: left oid 39 (ObjNum 38 snap 0 seq_num 38) 2024-04-18T23:10:58.483 INFO:tasks.rados.rados.0.smithi142.stdout:39: done (15 left) 2024-04-18T23:10:58.483 INFO:tasks.rados.rados.0.smithi142.stdout:51: copy_from oid 2 from oid 11 current snap is 0 2024-04-18T23:10:58.483 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.487 INFO:tasks.rados.rados.0.smithi142.stdout:43: finishing write tid 3 to smithi14230322-43 2024-04-18T23:10:58.487 INFO:tasks.rados.rados.0.smithi142.stdout:43: oid 43 updating version 0 to 2 2024-04-18T23:10:58.487 INFO:tasks.rados.rados.0.smithi142.stdout:43: oid 43 version 2 is already newer than 1 2024-04-18T23:10:58.487 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 43 v 2 (ObjNum 42 snap 0 seq_num 42) dirty exists 2024-04-18T23:10:58.487 INFO:tasks.rados.rados.0.smithi142.stdout:43: left oid 43 (ObjNum 42 snap 0 seq_num 42) 2024-04-18T23:10:58.487 INFO:tasks.rados.rados.0.smithi142.stdout:43: done (15 left) 2024-04-18T23:10:58.487 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 25 current snap is 0 2024-04-18T23:10:58.487 INFO:tasks.rados.rados.0.smithi142.stdout:52: seq_num 50 ranges {409600=475136,884736=376832} 2024-04-18T23:10:58.493 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:58.489+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:10:58.501 INFO:tasks.rados.rados.0.smithi142.stdout:52: writing smithi14230322-25 from 409600 to 884736 tid 1 2024-04-18T23:10:58.508 INFO:tasks.rados.rados.0.smithi142.stdout:52: writing smithi14230322-25 from 884736 to 1261568 tid 2 2024-04-18T23:10:58.508 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:10:58.546 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:58.546 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:58.561 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:10:58.579 INFO:tasks.rados.rados.0.smithi142.stdout:40: finishing write tid 2 to smithi14230322-40 2024-04-18T23:10:58.580 INFO:tasks.rados.rados.0.smithi142.stdout:42: finishing write tid 1 to smithi14230322-42 2024-04-18T23:10:58.581 INFO:tasks.rados.rados.0.smithi142.stdout:42: finishing write tid 2 to smithi14230322-42 2024-04-18T23:10:58.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-04-18T23:10:58.632 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:58.630+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:10:58.656 INFO:tasks.rados.rados.0.smithi142.stdout:46: finishing write tid 2 to smithi14230322-46 2024-04-18T23:10:58.677 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:58.678 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:38: finishing write tid 1 to smithi14230322-38 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:38: finishing write tid 2 to smithi14230322-38 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:32: finishing write tid 3 to smithi14230322-32 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:32: oid 32 updating version 0 to 6 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:32: oid 32 version 6 is already newer than 5 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 32 v 6 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:32: left oid 32 (ObjNum 31 snap 0 seq_num 31) 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:45: finishing write tid 1 to smithi14230322-45 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:32: done (15 left) 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:53: rmattr oid 30 current snap is 0 2024-04-18T23:10:58.680 INFO:tasks.rados.rados.0.smithi142.stdout:53: done (15 left) 2024-04-18T23:10:58.681 INFO:tasks.rados.rados.0.smithi142.stdout:54: delete oid 3 current snap is 0 2024-04-18T23:10:58.681 INFO:tasks.rados.rados.0.smithi142.stdout:41: finishing write tid 2 to smithi14230322-41 2024-04-18T23:10:58.681 INFO:tasks.rados.rados.0.smithi142.stdout:45: finishing write tid 2 to smithi14230322-45 2024-04-18T23:10:58.689 INFO:tasks.rados.rados.0.smithi142.stdout:40: finishing write tid 3 to smithi14230322-40 2024-04-18T23:10:58.689 INFO:tasks.rados.rados.0.smithi142.stdout:40: oid 40 updating version 0 to 9 2024-04-18T23:10:58.689 INFO:tasks.rados.rados.0.smithi142.stdout:40: oid 40 updating version 9 to 10 2024-04-18T23:10:58.689 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 40 v 10 (ObjNum 39 snap 0 seq_num 39) dirty exists 2024-04-18T23:10:58.689 INFO:tasks.rados.rados.0.smithi142.stdout:40: left oid 40 (ObjNum 39 snap 0 seq_num 39) 2024-04-18T23:10:58.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:10:58.695 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:58.693+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:10:58.780 INFO:tasks.rados.rados.0.smithi142.stdout:44: finishing write tid 1 to smithi14230322-44 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:36: finishing write tid 3 to smithi14230322-36 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:36: oid 36 updating version 0 to 8 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:36: oid 36 version 8 is already newer than 7 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 36 v 8 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:36: left oid 36 (ObjNum 35 snap 0 seq_num 35) 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:37: finishing write tid 3 to smithi14230322-37 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:37: oid 37 updating version 0 to 10 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:37: oid 37 version 10 is already newer than 9 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 37 v 10 (ObjNum 36 snap 0 seq_num 36) dirty exists 2024-04-18T23:10:58.782 INFO:tasks.rados.rados.0.smithi142.stdout:37: left oid 37 (ObjNum 36 snap 0 seq_num 36) 2024-04-18T23:10:58.795 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:10:58.793+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:10:58.804 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-04-18T23:10:58.856 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:58.856 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:58.859 INFO:tasks.rados.rados.0.smithi142.stdout:52: finishing write tid 1 to smithi14230322-25 2024-04-18T23:10:58.860 INFO:tasks.rados.rados.0.smithi142.stdout:46: finishing write tid 3 to smithi14230322-46 2024-04-18T23:10:58.861 INFO:tasks.rados.rados.0.smithi142.stdout:46: oid 46 updating version 0 to 6 2024-04-18T23:10:58.861 INFO:tasks.rados.rados.0.smithi142.stdout:46: oid 46 version 6 is already newer than 5 2024-04-18T23:10:58.861 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 46 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-18T23:10:58.861 INFO:tasks.rados.rados.0.smithi142.stdout:46: left oid 46 (ObjNum 45 snap 0 seq_num 45) 2024-04-18T23:10:58.871 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:10:58.881 INFO:tasks.rados.rados.0.smithi142.stdout:44: finishing write tid 2 to smithi14230322-44 2024-04-18T23:10:58.883 INFO:tasks.rados.rados.0.smithi142.stdout:48: finishing write tid 1 to smithi14230322-48 2024-04-18T23:10:58.890 INFO:tasks.rados.rados.0.smithi142.stdout:51: finishing copy_from racing read to smithi14230322-2 2024-04-18T23:10:58.916 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:58.914+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:10:58.968 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-04-18T23:10:58.984 INFO:tasks.rados.rados.0.smithi142.stdout:48: finishing write tid 2 to smithi14230322-48 2024-04-18T23:10:58.984 INFO:tasks.rados.rados.0.smithi142.stdout:52: finishing write tid 2 to smithi14230322-25 2024-04-18T23:10:58.984 INFO:tasks.rados.rados.0.smithi142.stdout:52: finishing write tid 3 to smithi14230322-25 2024-04-18T23:10:58.989 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:58.989 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:58.996 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:10:58.993+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:10:59.005 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:10:59.083 INFO:tasks.rados.rados.0.smithi142.stdout:38: finishing write tid 3 to smithi14230322-38 2024-04-18T23:10:59.083 INFO:tasks.rados.rados.0.smithi142.stdout:38: oid 38 updating version 0 to 11 2024-04-18T23:10:59.083 INFO:tasks.rados.rados.0.smithi142.stdout:38: oid 38 updating version 11 to 12 2024-04-18T23:10:59.083 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 38 v 12 (ObjNum 37 snap 0 seq_num 37) dirty exists 2024-04-18T23:10:59.083 INFO:tasks.rados.rados.0.smithi142.stdout:38: left oid 38 (ObjNum 37 snap 0 seq_num 37) 2024-04-18T23:10:59.083 INFO:tasks.rados.rados.0.smithi142.stdout:49: finishing write tid 1 to smithi14230322-49 2024-04-18T23:10:59.083 INFO:tasks.rados.rados.0.smithi142.stdout:49: finishing write tid 2 to smithi14230322-49 2024-04-18T23:10:59.086 INFO:tasks.rados.rados.0.smithi142.stdout:44: finishing write tid 3 to smithi14230322-44 2024-04-18T23:10:59.086 INFO:tasks.rados.rados.0.smithi142.stdout:44: oid 44 updating version 0 to 7 2024-04-18T23:10:59.086 INFO:tasks.rados.rados.0.smithi142.stdout:44: oid 44 updating version 7 to 8 2024-04-18T23:10:59.086 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 44 v 8 (ObjNum 43 snap 0 seq_num 43) dirty exists 2024-04-18T23:10:59.087 INFO:tasks.rados.rados.0.smithi142.stdout:44: left oid 44 (ObjNum 43 snap 0 seq_num 43) 2024-04-18T23:10:59.097 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:59.093+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:10:59.149 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-04-18T23:10:59.159 INFO:tasks.rados.rados.0.smithi142.stdout:52: finishing write tid 4 to smithi14230322-25 2024-04-18T23:10:59.159 INFO:tasks.rados.rados.0.smithi142.stdout:52: oid 25 updating version 0 to 7 2024-04-18T23:10:59.159 INFO:tasks.rados.rados.0.smithi142.stdout:52: oid 25 updating version 7 to 9 2024-04-18T23:10:59.160 INFO:tasks.rados.rados.0.smithi142.stdout:52: oid 25 version 9 is already newer than 8 2024-04-18T23:10:59.160 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 25 v 9 (ObjNum 50 snap 0 seq_num 50) dirty exists 2024-04-18T23:10:59.160 INFO:tasks.rados.rados.0.smithi142.stdout:52: left oid 25 (ObjNum 50 snap 0 seq_num 50) 2024-04-18T23:10:59.167 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:59.167 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:59.182 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:10:59.183 INFO:tasks.rados.rados.0.smithi142.stdout:49: finishing write tid 3 to smithi14230322-49 2024-04-18T23:10:59.184 INFO:tasks.rados.rados.0.smithi142.stdout:45: finishing write tid 3 to smithi14230322-45 2024-04-18T23:10:59.184 INFO:tasks.rados.rados.0.smithi142.stdout:45: oid 45 updating version 0 to 8 2024-04-18T23:10:59.184 INFO:tasks.rados.rados.0.smithi142.stdout:45: oid 45 version 8 is already newer than 7 2024-04-18T23:10:59.184 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 45 v 8 (ObjNum 44 snap 0 seq_num 44) dirty exists 2024-04-18T23:10:59.184 INFO:tasks.rados.rados.0.smithi142.stdout:45: left oid 45 (ObjNum 44 snap 0 seq_num 44) 2024-04-18T23:10:59.190 INFO:tasks.rados.rados.0.smithi142.stdout:42: finishing write tid 3 to smithi14230322-42 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:42: oid 42 updating version 0 to 11 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:42: oid 42 updating version 11 to 12 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 42 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:42: left oid 42 (ObjNum 41 snap 0 seq_num 41) 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:36: done (15 left) 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:37: done (14 left) 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:38: done (13 left) 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:40: done (12 left) 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:42: done (11 left) 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:44: done (10 left) 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:45: done (9 left) 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:46: done (8 left) 2024-04-18T23:10:59.191 INFO:tasks.rados.rados.0.smithi142.stdout:52: done (7 left) 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:54: done (6 left) 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:55: read oid 31 snap -1 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:55: expect (ObjNum 30 snap 0 seq_num 30) 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:56: rollback oid 38 current snap is 0 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:56: done (7 left) 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:57: read oid 46 snap -1 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:57: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:58: copy_from oid 3 from oid 30 current snap is 0 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:59: rollback oid 15 current snap is 0 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:59: done (9 left) 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:60: copy_from oid 26 from oid 20 current snap is 0 2024-04-18T23:10:59.192 INFO:tasks.rados.rados.0.smithi142.stdout:61: delete oid 40 current snap is 0 2024-04-18T23:10:59.198 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:59.197+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:10:59.259 INFO:tasks.rados.rados.0.smithi142.stdout:47: finishing write tid 1 to smithi14230322-47 2024-04-18T23:10:59.259 INFO:tasks.rados.rados.0.smithi142.stdout:47: finishing write tid 2 to smithi14230322-47 2024-04-18T23:10:59.259 INFO:tasks.rados.rados.0.smithi142.stdout:51: finishing copy_from to smithi14230322-2 2024-04-18T23:10:59.259 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 2 v 6 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-18T23:10:59.260 INFO:tasks.rados.rados.0.smithi142.stdout:50: finishing write tid 1 to smithi14230322-50 2024-04-18T23:10:59.260 INFO:tasks.rados.rados.0.smithi142.stdout:50: finishing write tid 2 to smithi14230322-50 2024-04-18T23:10:59.260 INFO:tasks.rados.rados.0.smithi142.stdout:50: finishing write tid 3 to smithi14230322-50 2024-04-18T23:10:59.292 INFO:tasks.rados.rados.0.smithi142.stdout:41: finishing write tid 3 to smithi14230322-41 2024-04-18T23:10:59.292 INFO:tasks.rados.rados.0.smithi142.stdout:41: oid 41 updating version 0 to 6 2024-04-18T23:10:59.292 INFO:tasks.rados.rados.0.smithi142.stdout:41: oid 41 version 6 is already newer than 5 2024-04-18T23:10:59.292 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 41 v 6 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-18T23:10:59.292 INFO:tasks.rados.rados.0.smithi142.stdout:41: left oid 41 (ObjNum 40 snap 0 seq_num 40) 2024-04-18T23:10:59.293 INFO:tasks.rados.rados.0.smithi142.stdout:48: finishing write tid 3 to smithi14230322-48 2024-04-18T23:10:59.298 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:10:59.294+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:10:59.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-04-18T23:10:59.341 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:59.341 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:59.355 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:49: finishing write tid 4 to smithi14230322-49 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:49: oid 49 updating version 0 to 10 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:49: oid 49 updating version 10 to 11 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:49: oid 49 version 11 is already newer than 9 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 49 v 11 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:49: left oid 49 (ObjNum 48 snap 0 seq_num 48) 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:47: finishing write tid 3 to smithi14230322-47 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:47: oid 47 updating version 0 to 14 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:47: oid 47 version 14 is already newer than 13 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 47 v 14 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-18T23:10:59.385 INFO:tasks.rados.rados.0.smithi142.stdout:47: left oid 47 (ObjNum 46 snap 0 seq_num 46) 2024-04-18T23:10:59.398 INFO:tasks.rados.rados.0.smithi142.stdout:57: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-18T23:10:59.399 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:59.401+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:10:59.453 INFO:tasks.rados.rados.0.smithi142.stdout:60: finishing copy_from racing read to smithi14230322-26 2024-04-18T23:10:59.485 INFO:tasks.rados.rados.0.smithi142.stdout:50: finishing write tid 4 to smithi14230322-50 2024-04-18T23:10:59.486 INFO:tasks.rados.rados.0.smithi142.stdout:50: oid 50 updating version 0 to 17 2024-04-18T23:10:59.486 INFO:tasks.rados.rados.0.smithi142.stdout:50: oid 50 version 17 is already newer than 15 2024-04-18T23:10:59.486 INFO:tasks.rados.rados.0.smithi142.stdout:50: oid 50 version 17 is already newer than 16 2024-04-18T23:10:59.486 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 50 v 17 (ObjNum 49 snap 0 seq_num 49) dirty exists 2024-04-18T23:10:59.486 INFO:tasks.rados.rados.0.smithi142.stdout:50: left oid 50 (ObjNum 49 snap 0 seq_num 49) 2024-04-18T23:10:59.489 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-04-18T23:10:59.496 INFO:tasks.rados.rados.0.smithi142.stdout:55: expect (ObjNum 30 snap 0 seq_num 30) 2024-04-18T23:10:59.500 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:10:59.497+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:10:59.508 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:59.508 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:59.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:10:59.525 INFO:tasks.rados.rados.0.smithi142.stdout:48: finishing write tid 4 to smithi14230322-48 2024-04-18T23:10:59.525 INFO:tasks.rados.rados.0.smithi142.stdout:48: oid 48 updating version 0 to 11 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:48: oid 48 version 11 is already newer than 10 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:48: oid 48 version 11 is already newer than 9 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 48 v 11 (ObjNum 47 snap 0 seq_num 47) dirty exists 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:48: left oid 48 (ObjNum 47 snap 0 seq_num 47) 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:58: finishing copy_from racing read to smithi14230322-3 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:41: done (10 left) 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:47: done (9 left) 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:48: done (8 left) 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:49: done (7 left) 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:50: done (6 left) 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:51: done (5 left) 2024-04-18T23:10:59.526 INFO:tasks.rados.rados.0.smithi142.stdout:55: done (4 left) 2024-04-18T23:10:59.527 INFO:tasks.rados.rados.0.smithi142.stdout:57: done (3 left) 2024-04-18T23:10:59.527 INFO:tasks.rados.rados.0.smithi142.stdout:61: done (2 left) 2024-04-18T23:10:59.527 INFO:tasks.rados.rados.0.smithi142.stdout:62: read oid 33 snap -1 2024-04-18T23:10:59.527 INFO:tasks.rados.rados.0.smithi142.stdout:62: expect (ObjNum 32 snap 0 seq_num 32) 2024-04-18T23:10:59.527 INFO:tasks.rados.rados.0.smithi142.stdout:63: snap_create 2024-04-18T23:10:59.527 INFO:tasks.rados.rados.0.smithi142.stdout:60: finishing copy_from to smithi14230322-26 2024-04-18T23:10:59.527 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 26 v 2 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-04-18T23:10:59.558 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T23:10:59.600 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:10:59.599+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:10:59.658 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-04-18T23:10:59.667 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:10:59.677 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:59.677 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:59.693 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:10:59.701 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:10:59.697+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:10:59.734 INFO:tasks.rados.rados.0.smithi142.stdout:62: expect (ObjNum 32 snap 0 seq_num 32) 2024-04-18T23:10:59.802 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:10:59.801+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:10:59.807 INFO:tasks.rados.rados.0.smithi142.stdout:58: finishing copy_from to smithi14230322-3 2024-04-18T23:10:59.807 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 3 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-18T23:10:59.826 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 reset_purged_snaps_last 2024-04-18T23:10:59.848 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:10:59.848 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:10:59.862 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:10:59.902 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:10:59.901+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:10:59.933 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:10:59.933 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:10:59.957 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:10:59.957 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:10:59.984 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:10:59.985 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:10:59.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 reset_purged_snaps_last 2024-04-18T23:10:59.993 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:10:59.993 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:10:59.993 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:10:59.993 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:00.003 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:00.005+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:00.005 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:00.005 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:00.007 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:11:00.020 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:00.020 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:00.036 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:00.037 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:00.056 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:00.056 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:00.075 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:00.076 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:00.092 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:00.093 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:00.103 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:00.105+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:00.111 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:00.111 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:00.132 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:00.132 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:00.139 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 reset_purged_snaps_last 2024-04-18T23:11:00.152 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:00.152 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:00.168 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:00.168 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:00.175 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:00.175 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:00.175 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:00.175 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:00.175 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:00.187 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:00.187 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:00.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:11:00.204 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:00.203+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:00.207 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:00.207 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:00.226 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:00.226 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:00.298 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 reset_purged_snaps_last 2024-04-18T23:11:00.344 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:00.342+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:00.354 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:00.354 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:00.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:00.354 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:00.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.354 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.2s5 33/31 ECSubWrite(tid=157, reqid=client.4568.0:10, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.520022+0000", 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.8339441980000002, 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0087933809999999994, 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520022+0000", 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520022+0000", 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520023+0000", 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1299999999999998e-07 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520115+0000", 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2406999999999998e-05 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520161+0000", 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5603999999999999e-05 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520190+0000", 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9187999999999999e-05 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.523272+0000", 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030820330000000001 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.523296+0000", 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4321999999999999e-05 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528726+0000", 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0054300629999999997 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528815+0000", 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8850999999999993e-05 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.358 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/31 ECSubWrite(tid=2, reqid=client.4568.0:19, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.8259630480000002, 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19915530100000001, 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:00.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528004+0000", 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528243+0000", 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239244 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528271+0000", 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7535e-05 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528293+0000", 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2395000000000002e-05 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.626035+0000", 2024-04-18T23:11:00.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.097742512000000004 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.626059+0000", 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3739000000000001e-05 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727090+0000", 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101030651 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727158+0000", 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8415000000000001e-05 2024-04-18T23:11:00.362 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 34/31 ECSubWrite(tid=0, reqid=client.4568.0:7, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.528057+0000", 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.8259090950000001, 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035481419999999998, 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528057+0000", 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528057+0000", 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528058+0000", 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1299999999999999e-07 2024-04-18T23:11:00.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528178+0000", 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000120826 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528210+0000", 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1568e-05 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528234+0000", 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4175000000000001e-05 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528392+0000", 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015808900000000001 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528414+0000", 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1974999999999999e-05 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531518+0000", 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0031035619999999998 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531605+0000", 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7233999999999999e-05 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 34/31 ECSubWrite(tid=1, reqid=client.4568.0:8, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.530089+0000", 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.8238770510000002, 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022304099999999999, 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530089+0000", 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530089+0000", 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530091+0000", 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7060000000000001e-06 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530134+0000", 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2970999999999999e-05 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530188+0000", 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4410999999999998e-05 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530214+0000", 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6183999999999999e-05 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530385+0000", 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017049099999999999 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530410+0000", 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5575000000000002e-05 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.532254+0000", 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018436259999999999 2024-04-18T23:11:00.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.532319+0000", 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5445999999999996e-05 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 34/31 ECSubWrite(tid=0, reqid=client.4568.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.535304+0000", 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.8186623590000002, 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002377947, 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.371 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535304+0000", 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535304+0000", 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535305+0000", 2024-04-18T23:11:00.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4399999999999998e-07 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535613+0000", 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00030811499999999997 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535654+0000", 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1634000000000002e-05 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535679+0000", 2024-04-18T23:11:00.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4569000000000001e-05 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535812+0000", 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013308999999999999 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535836+0000", 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4127000000000001e-05 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537577+0000", 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001740662 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537682+0000", 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104806 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:00.375 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.8160784830000001, 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103172215, 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537889+0000", 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839999999999999e-06 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537921+0000", 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2360999999999997e-05 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537948+0000", 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6421e-05 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537963+0000", 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5218999999999999e-05 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638148+0000", 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100185015 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638163+0000", 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4793e-05 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.640970+0000", 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028075439999999999 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.641060+0000", 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9377999999999997e-05 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 34/31 ECSubWrite(tid=0, reqid=client.4568.0:43, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.621642+0000", 2024-04-18T23:11:00.379 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.73232381, 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002593429, 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.621642+0000", 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.621642+0000", 2024-04-18T23:11:00.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.621643+0000", 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2699999999999999e-06 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.621766+0000", 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000122819 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.621809+0000", 2024-04-18T23:11:00.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2589999999999997e-05 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.621834+0000", 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5119000000000001e-05 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622055+0000", 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000220855 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622076+0000", 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1631e-05 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.624154+0000", 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020776010000000001 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.624236+0000", 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1544000000000004e-05 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.383 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.2s5 33/31 ECSubWrite(tid=158, reqid=client.4568.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.622803+0000", 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7311633080000002, 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022328399999999998, 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622803+0000", 2024-04-18T23:11:00.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622803+0000", 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622805+0000", 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9e-06 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622850+0000", 2024-04-18T23:11:00.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5800000000000002e-05 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622883+0000", 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2458e-05 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622907+0000", 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4085000000000001e-05 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623072+0000", 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000165126 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623098+0000", 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5828000000000001e-05 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.624837+0000", 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001739343 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625036+0000", 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001983 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.2s5 33/31 ECSubWrite(tid=159, reqid=client.4568.0:25, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.622957+0000", 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7310087379999999, 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002911004, 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622957+0000", 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622957+0000", 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622958+0000", 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0529999999999999e-06 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623083+0000", 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012421200000000001 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623107+0000", 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4377999999999999e-05 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623129+0000", 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1645e-05 2024-04-18T23:11:00.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623619+0000", 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049091399999999996 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623641+0000", 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1294999999999999e-05 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625815+0000", 2024-04-18T23:11:00.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021744440000000002 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625868+0000", 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3062999999999997e-05 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.2s5 33/31 ECSubWrite(tid=160, reqid=client.4568.0:26, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.623173+0000", 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7307931519999999, 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101822397, 2024-04-18T23:11:00.392 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623173+0000", 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623173+0000", 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623174+0000", 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.16e-07 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623206+0000", 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2660999999999998e-05 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623229+0000", 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2481999999999999e-05 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723331+0000", 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010213699999999 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723530+0000", 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019928599999999999 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723553+0000", 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2442e-05 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724939+0000", 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013867300000000001 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724995+0000", 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5742999999999998e-05 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=0, reqid=client.4568.0:4, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.625070+0000", 2024-04-18T23:11:00.396 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7288956770000001, 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0059680189999999998, 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625070+0000", 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625070+0000", 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625071+0000", 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6400000000000002e-07 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625134+0000", 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3459000000000003e-05 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625178+0000", 2024-04-18T23:11:00.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3696999999999999e-05 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625202+0000", 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4258000000000001e-05 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625406+0000", 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020418300000000001 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625432+0000", 2024-04-18T23:11:00.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5235000000000001e-05 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.630957+0000", 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005524892 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.631038+0000", 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1730999999999994e-05 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:5, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.725877273, 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103548802, 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628091+0000", 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1129999999999999e-06 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628179+0000", 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7818000000000005e-05 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628219+0000", 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0373000000000001e-05 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628243+0000", 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3549000000000001e-05 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628766+0000", 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052308499999999996 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628791+0000", 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4984999999999999e-05 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731546+0000", 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102755535 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731638+0000", 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1343999999999998e-05 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.404 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=2, reqid=client.4568.0:22, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7256651170000001, 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103381863, 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:00.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628302+0000", 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628476+0000", 2024-04-18T23:11:00.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017366199999999999 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628499+0000", 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3025000000000001e-05 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628519+0000", 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0194999999999999e-05 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629371+0000", 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085246300000000003 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629394+0000", 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3311000000000001e-05 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731650+0000", 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10225558899999999 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731683+0000", 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2654999999999998e-05 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=3, reqid=client.4568.0:23, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7249679269999998, 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103947704, 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.409 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629005+0000", 2024-04-18T23:11:00.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7089999999999996e-06 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629040+0000", 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5133999999999999e-05 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629064+0000", 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4422000000000001e-05 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629079+0000", 2024-04-18T23:11:00.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4949e-05 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731181+0000", 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102101339 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731199+0000", 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8175000000000001e-05 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.732877+0000", 2024-04-18T23:11:00.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016778050000000001 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.732946+0000", 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9171000000000006e-05 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.0s3 34/31 ECSubWrite(tid=2, reqid=client.4568.0:46, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.636098+0000", 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.7178678249999999, 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023930359999999999, 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.413 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636098+0000", 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636098+0000", 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636101+0000", 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3829999999999999e-06 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636357+0000", 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025627000000000001 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636393+0000", 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6491e-05 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636422+0000", 2024-04-18T23:11:00.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8668000000000001e-05 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636564+0000", 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014215699999999999 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636586+0000", 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1401999999999999e-05 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638411+0000", 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00182507 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638491+0000", 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0594999999999997e-05 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.417 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.2s5 34/31 ECSubWrite(tid=161, reqid=client.4568.0:31, at_version=34'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.723408+0000", 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.630558298, 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00704603, 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723408+0000", 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723408+0000", 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723408+0000", 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4300000000000003e-07 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723539+0000", 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000130782 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723568+0000", 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9420999999999998e-05 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723591+0000", 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2965000000000001e-05 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724103+0000", 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000511602 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724128+0000", 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5275000000000001e-05 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.730366+0000", 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062380409999999997 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.730454+0000", 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7400999999999999e-05 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.2s5 34/31 ECSubWrite(tid=162, reqid=client.4568.0:32, at_version=34'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.723634+0000", 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.6303315400000002, 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0068638589999999999, 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723634+0000", 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723634+0000", 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723635+0000", 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8000000000000004e-07 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723715+0000", 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9938000000000006e-05 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723741+0000", 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5918000000000002e-05 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723760+0000", 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8836000000000001e-05 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725084+0000", 2024-04-18T23:11:00.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013243440000000001 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725107+0000", 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2291999999999999e-05 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.730466+0000", 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0053592400000000004 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.730498+0000", 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2410999999999999e-05 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:44, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.723998+0000", 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.6299674980000001, 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035512550000000001, 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.426 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723998+0000", 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723998+0000", 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724000+0000", 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1200000000000001e-06 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724039+0000", 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9028e-05 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724064+0000", 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5791000000000001e-05 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724084+0000", 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9281999999999999e-05 2024-04-18T23:11:00.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724226+0000", 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142194 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724240+0000", 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3619e-05 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727500+0000", 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032601539999999999 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727550+0000", 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0067e-05 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 34/31 ECSubWrite(tid=4, reqid=client.4568.0:8, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.736334+0000", 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.6176320070000001, 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013388160000000001, 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.430 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736334+0000", 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736334+0000", 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736336+0000", 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8029999999999999e-06 2024-04-18T23:11:00.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736362+0000", 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6577999999999999e-05 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736389+0000", 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6852e-05 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736414+0000", 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4688000000000001e-05 2024-04-18T23:11:00.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736592+0000", 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000178178 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736620+0000", 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8382e-05 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737619+0000", 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099838599999999998 2024-04-18T23:11:00.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737673+0000", 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3949000000000001e-05 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.434 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:00.436 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:11:00.436 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:00.433+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:11:00.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 reset_purged_snaps_last 2024-04-18T23:11:00.534 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-04-18T23:11:00.537 INFO:tasks.rados.rados.0.smithi142.stdout:58: done (3 left) 2024-04-18T23:11:00.537 INFO:tasks.rados.rados.0.smithi142.stdout:60: done (2 left) 2024-04-18T23:11:00.537 INFO:tasks.rados.rados.0.smithi142.stdout:62: done (1 left) 2024-04-18T23:11:00.537 INFO:tasks.rados.rados.0.smithi142.stdout:63: done (0 left) 2024-04-18T23:11:00.537 INFO:tasks.rados.rados.0.smithi142.stdout:64: copy_from oid 15 from oid 2 current snap is 1 2024-04-18T23:11:00.537 INFO:tasks.rados.rados.0.smithi142.stdout:65: read oid 30 snap -1 2024-04-18T23:11:00.537 INFO:tasks.rados.rados.0.smithi142.stdout:65: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-18T23:11:00.537 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 23 current snap is 1 2024-04-18T23:11:00.537 INFO:tasks.rados.rados.0.smithi142.stdout:66: seq_num 53 ranges {376832=524288,901120=376832} 2024-04-18T23:11:00.537 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:00.537+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:00.550 INFO:tasks.rados.rados.0.smithi142.stdout:66: writing smithi14230322-23 from 376832 to 901120 tid 1 2024-04-18T23:11:00.556 INFO:tasks.rados.rados.0.smithi142.stdout:66: writing smithi14230322-23 from 901120 to 1277952 tid 2 2024-04-18T23:11:00.556 INFO:tasks.rados.rados.0.smithi142.stdout:67: rollback oid 10 current snap is 1 2024-04-18T23:11:00.556 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 10 to 0 2024-04-18T23:11:00.556 INFO:tasks.rados.rados.0.smithi142.stdout:68: setattr oid 32 current snap is 1 2024-04-18T23:11:00.558 INFO:tasks.rados.rados.0.smithi142.stdout:69: copy_from oid 7 from oid 17 current snap is 1 2024-04-18T23:11:00.558 INFO:tasks.rados.rados.0.smithi142.stdout:70: read oid 20 snap 0 2024-04-18T23:11:00.558 INFO:tasks.rados.rados.0.smithi142.stdout:70: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-18T23:11:00.558 INFO:tasks.rados.rados.0.smithi142.stdout:71: copy_from oid 24 from oid 29 current snap is 1 2024-04-18T23:11:00.558 INFO:tasks.rados.rados.0.smithi142.stdout:72: rollback oid 6 current snap is 1 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 6 to 0 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:73: read oid 13 snap 0 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:73: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:74: rmattr oid 4 current snap is 1 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:74: done (10 left) 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:75: copy_from oid 48 from oid 1 current snap is 1 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:76: read oid 27 snap -1 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:76: expect (ObjNum 26 snap 0 seq_num 26) 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:77: copy_from oid 28 from oid 34 current snap is 1 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:78: read oid 43 snap -1 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:78: expect (ObjNum 42 snap 0 seq_num 42) 2024-04-18T23:11:00.559 INFO:tasks.rados.rados.0.smithi142.stdout:79: setattr oid 14 current snap is 1 2024-04-18T23:11:00.560 INFO:tasks.rados.rados.0.smithi142.stdout:80: snap_create 2024-04-18T23:11:00.567 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:00.567 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:176 3.8s0 3.8b4d0908 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.030650883, 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030691427, 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "op must wait for map", 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 176 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539638+0000", 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3829999999999999e-06 2024-04-18T23:11:00.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539651+0000", 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2549e-05 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539675+0000", 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4413000000000002e-05 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539689+0000", 2024-04-18T23:11:00.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4161999999999999e-05 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539831+0000", 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014226500000000001 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op must wait for map", 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539865+0000", 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3185000000000002e-05 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:177 3.8s0 3.8b4d0908 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.030568251000000001, 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.030727655, 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "waiting_for_map not empty", 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 177 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539719+0000", 2024-04-18T23:11:00.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.37e-07 2024-04-18T23:11:00.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539723+0000", 2024-04-18T23:11:00.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1620000000000001e-06 2024-04-18T23:11:00.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539733+0000", 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4630000000000007e-06 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539739+0000", 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7880000000000001e-06 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540029+0000", 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029001100000000001 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540050+0000", 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1033e-05 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:207 3.8s0 3.d6059838 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0089520840000000008, 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092125620000000005, 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.576 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "waiting_for_map not empty", 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 207 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:00.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561336+0000", 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.638e-06 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561354+0000", 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7413000000000001e-05 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561372+0000", 2024-04-18T23:11:00.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8003e-05 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561383+0000", 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1425e-05 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561495+0000", 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111661 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561506+0000", 2024-04-18T23:11:00.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.113e-05 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:208 3.8s0 3.d6059838 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.0088789950000000006, 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0092369640000000003, 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "waiting_for_map not empty", 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:00.580 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 208 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561411+0000", 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9179999999999998e-06 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561421+0000", 2024-04-18T23:11:00.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4599999999999992e-06 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561427+0000", 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561582+0000", 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000154652 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:00.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561592+0000", 2024-04-18T23:11:00.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.004e-05 2024-04-18T23:11:00.584 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.584 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.584 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.584 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.584 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-04-18T23:11:00.584 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 4 2024-04-18T23:11:00.584 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:00.585 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:11:00.639 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 reset_purged_snaps_last 2024-04-18T23:11:00.651 INFO:tasks.rados.rados.0.smithi142.stdout:65: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-18T23:11:00.678 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:00.681+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:00.739 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:00.737+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:00.754 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:00.754 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:00.754 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:00.754 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:00.754 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:00.767 INFO:tasks.rados.rados.0.smithi142.stdout:71: finishing copy_from racing read to smithi14230322-24 2024-04-18T23:11:00.771 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:11:00.794 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 reset_purged_snaps_last 2024-04-18T23:11:00.840 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:00.841+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:00.861 INFO:tasks.rados.rados.0.smithi142.stdout:77: finishing copy_from racing read to smithi14230322-28 2024-04-18T23:11:00.863 INFO:tasks.rados.rados.0.smithi142.stdout:64: finishing copy_from racing read to smithi14230322-15 2024-04-18T23:11:00.864 INFO:tasks.rados.rados.0.smithi142.stdout:70: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-18T23:11:00.938 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 reset_purged_snaps_last 2024-04-18T23:11:00.939 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:00.939 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.1s3 33/31 ECSubWrite(tid=0, reqid=client.4568.0:1, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.518959+0000", 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4207010770000004, 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002259858, 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518959+0000", 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518959+0000", 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518959+0000", 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7999999999999995e-07 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.519055+0000", 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5847000000000006e-05 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.519087+0000", 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2357999999999997e-05 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.519112+0000", 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4814999999999999e-05 2024-04-18T23:11:00.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.519279+0000", 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000166389 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.519303+0000", 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4794999999999999e-05 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521002+0000", 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001698282 2024-04-18T23:11:00.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521219+0000", 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021679199999999999 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 33/31 ECSubWrite(tid=1, reqid=client.4568.0:16, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.520061+0000", 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4195988509999999, 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10251751100000001, 2024-04-18T23:11:00.944 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520061+0000", 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520061+0000", 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520063+0000", 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.95e-06 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520209+0000", 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014583700000000001 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520243+0000", 2024-04-18T23:11:00.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4334e-05 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520266+0000", 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3118000000000001e-05 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.620586+0000", 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100320109 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.620600+0000", 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3453e-05 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622489+0000", 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018891730000000001 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622578+0000", 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9536999999999993e-05 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.1s3 33/31 ECSubWrite(tid=3, reqid=client.4568.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526794+0000", 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4128658989999998, 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025200470000000001, 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526794+0000", 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526794+0000", 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526795+0000", 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0139999999999999e-06 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526829+0000", 2024-04-18T23:11:00.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4155000000000001e-05 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526866+0000", 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6742999999999997e-05 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526889+0000", 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3104000000000001e-05 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527236+0000", 2024-04-18T23:11:00.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034734499999999999 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527258+0000", 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1889999999999999e-05 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.529213+0000", 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019547319999999998 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.529314+0000", 2024-04-18T23:11:00.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010106399999999999 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 33/31 ECSubWrite(tid=4, reqid=client.4568.0:17, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526938+0000", 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4127222369999997, 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.096524838000000002, 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.953 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526938+0000", 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526938+0000", 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526938+0000", 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1699999999999997e-07 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526977+0000", 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8893999999999997e-05 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527003+0000", 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5953e-05 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527021+0000", 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7306999999999999e-05 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.621378+0000", 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.094356907000000004 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.621390+0000", 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2965e-05 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623353+0000", 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019620390000000001 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623462+0000", 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109856 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 33/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.623740+0000", 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.3159202680000002, 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100701047, 2024-04-18T23:11:00.957 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623740+0000", 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623740+0000", 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623752+0000", 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2886e-05 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623765+0000", 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3046e-05 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623796+0000", 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0793999999999997e-05 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723901+0000", 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010519 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724256+0000", 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000354149 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724441+0000", 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018498200000000001 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.960 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 33/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.723963+0000", 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2156970329999996, 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00045773599999999998, 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723963+0000", 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723963+0000", 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723965+0000", 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.137e-06 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723977+0000", 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2405999999999999e-05 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724009+0000", 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1850999999999997e-05 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724033+0000", 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3368000000000001e-05 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724373+0000", 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034080699999999999 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724420+0000", 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7166999999999998e-05 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.724364+0000", 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2152959829999999, 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000225508, 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724364+0000", 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724364+0000", 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724365+0000", 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.64e-06 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724375+0000", 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0720000000000006e-06 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724404+0000", 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9541999999999999e-05 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724424+0000", 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9701999999999999e-05 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724548+0000", 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012415899999999999 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724589+0000", 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1393000000000002e-05 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.724526+0000", 2024-04-18T23:11:00.967 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2151341640000002, 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10203172000000001, 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724526+0000", 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724526+0000", 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724527+0000", 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1799999999999999e-06 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724534+0000", 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3830000000000003e-06 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724556+0000", 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2317999999999999e-05 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724575+0000", 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8425999999999999e-05 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826497+0000", 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10192208699999999 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826557+0000", 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0325999999999998e-05 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.970 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 34/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.725568+0000", 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2140918940000001, 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101108719, 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725568+0000", 2024-04-18T23:11:00.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725568+0000", 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725570+0000", 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7e-06 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725579+0000", 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5959999999999993e-06 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725607+0000", 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7925000000000001e-05 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725623+0000", 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5719e-05 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826642+0000", 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101018832 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826677+0000", 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4947000000000003e-05 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 34/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.726115+0000", 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2135448340000003, 2024-04-18T23:11:00.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100686821, 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.726115+0000", 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.726115+0000", 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.726117+0000", 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.733e-06 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.726126+0000", 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5559999999999996e-06 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.726154+0000", 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7855e-05 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.726168+0000", 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4246e-05 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826768+0000", 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100600073 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826802+0000", 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3358000000000001e-05 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 34/31 ECSubWrite(tid=6, reqid=client.4568.0:17, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.733620+0000", 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.206039402, 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016504930000000001, 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.733620+0000", 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.733620+0000", 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.733622+0000", 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0030000000000001e-06 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.733650+0000", 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7355000000000002e-05 2024-04-18T23:11:00.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.733691+0000", 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0973999999999998e-05 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.733710+0000", 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9712e-05 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.733850+0000", 2024-04-18T23:11:00.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013950400000000001 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.733864+0000", 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4499e-05 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.735134+0000", 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012699409999999999 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.735271+0000", 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013650500000000001 2024-04-18T23:11:00.981 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.738762+0000", 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2008978209999999, 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.088098946999999997, 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.738762+0000", 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.738762+0000", 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.738763+0000", 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4300000000000001e-06 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.738772+0000", 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3159999999999997e-06 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.738797+0000", 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5558000000000002e-05 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.738812+0000", 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4256e-05 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826837+0000", 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.088025181999999993 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826861+0000", 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4204999999999999e-05 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.1s3 34/31 ECSubWrite(tid=10, reqid=client.4568.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.836126+0000", 2024-04-18T23:11:00.985 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1035334270000003, 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00207515, 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836126+0000", 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836126+0000", 2024-04-18T23:11:00.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836127+0000", 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8700000000000004e-07 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836146+0000", 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8604999999999999e-05 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836166+0000", 2024-04-18T23:11:00.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0472e-05 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836183+0000", 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6646e-05 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836301+0000", 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000118245 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836323+0000", 2024-04-18T23:11:00.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2158999999999999e-05 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.838090+0000", 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017667309999999999 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.838202+0000", 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011130500000000001 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.1s3 34/31 ECSubRead(tid=11, to_read={3:8af79e0a:::smithi14230322-1:head=0,4096,0}, subchunks={3:8af79e0a:::smithi14230322-1:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.836210+0000", 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.103450209, 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001651235, 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836210+0000", 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836210+0000", 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836210+0000", 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4600000000000005e-07 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836225+0000", 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4902999999999999e-05 2024-04-18T23:11:00.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836251+0000", 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5548e-05 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836273+0000", 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1931999999999999e-05 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.836768+0000", 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049574700000000005 2024-04-18T23:11:00.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.837861+0000", 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010925590000000001 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:177 3.8s0 3:1090b2d1:::smithi14230322-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.39994146400000002, 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32703084799999999, 2024-04-18T23:11:00.993 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 177 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539719+0000", 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.37e-07 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539723+0000", 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1620000000000001e-06 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:00.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539733+0000", 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4630000000000007e-06 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539739+0000", 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7880000000000001e-06 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540029+0000", 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029001100000000001 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:00.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540050+0000", 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1033e-05 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644678+0000", 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10462763999999999 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644714+0000", 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6202999999999999e-05 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866546+0000", 2024-04-18T23:11:00.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.22183165399999999 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866650+0000", 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104465 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866749+0000", 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8791999999999999e-05 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:00.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:208 3.8s0 3:1c19a06b:::smithi14230322-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.37825220799999998, 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29446849200000003, 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 208 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:00.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561411+0000", 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9179999999999998e-06 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561421+0000", 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4599999999999992e-06 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561427+0000", 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561582+0000", 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000154652 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561592+0000", 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.004e-05 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644937+0000", 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.083345591999999996 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644974+0000", 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6835000000000002e-05 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855704+0000", 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.21072980599999999 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855787+0000", 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3442000000000005e-05 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855876+0000", 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8660999999999997e-05 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/31 ECSubWrite(tid=54, reqid=client.4568.0:188, at_version=36'19, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.761593+0000", 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.17806698100000001, 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002541597, 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.761593+0000", 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.761593+0000", 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.761595+0000", 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3360000000000002e-06 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.761745+0000", 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000150135 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.761778+0000", 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2394000000000002e-05 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.761799+0000", 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1103000000000001e-05 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.761938+0000", 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013931100000000001 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.761956+0000", 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7632e-05 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764075+0000", 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021188130000000002 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764134+0000", 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9873000000000001e-05 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/31 ECSubWrite(tid=58, reqid=client.4568.0:196, at_version=36'21, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.762407+0000", 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.17725308000000001, 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017712400000000001, 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.008 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.762407+0000", 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.762407+0000", 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.762409+0000", 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8300000000000001e-06 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.762472+0000", 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3886000000000005e-05 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.762505+0000", 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2153999999999997e-05 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.762530+0000", 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5516000000000001e-05 2024-04-18T23:11:01.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763092+0000", 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00056216899999999995 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763098+0000", 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0050000000000001e-06 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764143+0000", 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010444689999999999 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764178+0000", 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5210999999999999e-05 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/31 ECSubWrite(tid=59, reqid=client.4568.0:197, at_version=36'22, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.763361+0000", 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.17629909099999999, 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00099885399999999997, 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.012 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763361+0000", 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763361+0000", 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763363+0000", 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8300000000000001e-06 2024-04-18T23:11:01.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763409+0000", 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6137000000000002e-05 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763440+0000", 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1167999999999997e-05 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763462+0000", 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2595999999999998e-05 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763638+0000", 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017578200000000001 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763656+0000", 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7439e-05 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764331+0000", 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00067486300000000005 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764360+0000", 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9039000000000001e-05 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/31 ECSubWrite(tid=60, reqid=client.4568.0:198, at_version=36'23, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.764060+0000", 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.17560004600000001, 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101243635, 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764060+0000", 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764060+0000", 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764061+0000", 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0729999999999999e-06 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764087+0000", 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6444999999999998e-05 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764104+0000", 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6622000000000001e-05 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.864187+0000", 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10008339199999999 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.864266+0000", 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9182000000000002e-05 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.864274+0000", 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4000000000000003e-06 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.865258+0000", 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000983755 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.865303+0000", 2024-04-18T23:11:01.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5766000000000002e-05 2024-04-18T23:11:01.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.020 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.020 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.020 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:01.022 INFO:tasks.rados.rados.0.smithi142.stdout:76: expect (ObjNum 26 snap 0 seq_num 26) 2024-04-18T23:11:01.022 INFO:tasks.rados.rados.0.smithi142.stdout:77: finishing copy_from to smithi14230322-28 2024-04-18T23:11:01.022 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 28 v 4 (ObjNum 33 snap 0 seq_num 33) dirty exists 2024-04-18T23:11:01.022 INFO:tasks.rados.rados.0.smithi142.stdout:66: finishing write tid 1 to smithi14230322-23 2024-04-18T23:11:01.022 INFO:tasks.rados.rados.0.smithi142.stdout:66: finishing write tid 2 to smithi14230322-23 2024-04-18T23:11:01.022 INFO:tasks.rados.rados.0.smithi142.stdout:73: expect (ObjNum 12 snap 0 seq_num 12) 2024-04-18T23:11:01.022 INFO:tasks.rados.rados.0.smithi142.stdout:71: finishing copy_from to smithi14230322-24 2024-04-18T23:11:01.022 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 24 v 4 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-18T23:11:01.023 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:11:01.023 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:01.019+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:01.052 INFO:tasks.rados.rados.0.smithi142.stdout:67: finishing rollback tid 0 to smithi14230322-10 2024-04-18T23:11:01.052 INFO:tasks.rados.rados.0.smithi142.stdout:67: finishing rollback tid 1 to smithi14230322-10 2024-04-18T23:11:01.052 INFO:tasks.rados.rados.0.smithi142.stdout:67: finishing rollback tid 2 to smithi14230322-10 2024-04-18T23:11:01.052 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 10 v 15 (ObjNum 9 snap 0 seq_num 9) dirty exists 2024-04-18T23:11:01.067 INFO:tasks.rados.rados.0.smithi142.stdout:66: finishing write tid 3 to smithi14230322-23 2024-04-18T23:11:01.067 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 32 v 19 (ObjNum 31 snap 0 seq_num 31) dirty exists 2024-04-18T23:11:01.067 INFO:tasks.rados.rados.0.smithi142.stdout:72: finishing rollback tid 0 to smithi14230322-6 2024-04-18T23:11:01.067 INFO:tasks.rados.rados.0.smithi142.stdout:64: finishing copy_from to smithi14230322-15 2024-04-18T23:11:01.067 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 15 v 6 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-18T23:11:01.070 INFO:tasks.rados.rados.0.smithi142.stdout:78: expect (ObjNum 42 snap 0 seq_num 42) 2024-04-18T23:11:01.122 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:01.121+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:01.150 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:01.150 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:01.150 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:01.151 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:01.164 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:11:01.168 INFO:tasks.rados.rados.0.smithi142.stdout:72: finishing rollback tid 1 to smithi14230322-6 2024-04-18T23:11:01.169 INFO:tasks.rados.rados.0.smithi142.stdout:75: finishing copy_from racing read to smithi14230322-48 2024-04-18T23:11:01.169 INFO:tasks.rados.rados.0.smithi142.stdout:72: finishing rollback tid 2 to smithi14230322-6 2024-04-18T23:11:01.169 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 6 v 23 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-18T23:11:01.169 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 14 v 25 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-18T23:11:01.171 INFO:tasks.rados.rados.0.smithi142.stdout:69: finishing copy_from racing read to smithi14230322-7 2024-04-18T23:11:01.172 INFO:tasks.rados.rados.0.smithi142.stdout:66: finishing write tid 4 to smithi14230322-23 2024-04-18T23:11:01.172 INFO:tasks.rados.rados.0.smithi142.stdout:66: oid 23 updating version 0 to 14 2024-04-18T23:11:01.172 INFO:tasks.rados.rados.0.smithi142.stdout:66: oid 23 updating version 14 to 15 2024-04-18T23:11:01.172 INFO:tasks.rados.rados.0.smithi142.stdout:66: oid 23 version 15 is already newer than 13 2024-04-18T23:11:01.173 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 23 v 15 (ObjNum 53 snap 1 seq_num 53) dirty exists 2024-04-18T23:11:01.173 INFO:tasks.rados.rados.0.smithi142.stdout:66: left oid 23 (ObjNum 53 snap 1 seq_num 53) 2024-04-18T23:11:01.223 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:01.223+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:01.286 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:01.286 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:01.286 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:01.286 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:01.286 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:01.299 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:11:01.324 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:01.322+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:01.371 INFO:tasks.rados.rados.0.smithi142.stdout:75: finishing copy_from to smithi14230322-48 2024-04-18T23:11:01.371 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 48 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-04-18T23:11:01.425 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:01.425 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.1s2 33/31 ECSubWrite(tid=0, reqid=client.4568.0:1, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.520429+0000", 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.9050409610000001, 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0061308580000000003, 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520429+0000", 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520429+0000", 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520430+0000", 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1799999999999999e-06 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520620+0000", 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000190027 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520654+0000", 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4171000000000002e-05 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520680+0000", 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5894999999999999e-05 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.522897+0000", 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022169009999999999 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.522931+0000", 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3515e-05 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526465+0000", 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035343079999999999 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526560+0000", 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4860999999999999e-05 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 33/31 ECSubWrite(tid=2, reqid=client.4568.0:19, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.525784+0000", 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.8996856209999997, 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.005084079, 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.430 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525784+0000", 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525784+0000", 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525785+0000", 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8700000000000005e-07 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525902+0000", 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000116995 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525925+0000", 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2872000000000001e-05 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525940+0000", 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5469e-05 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526099+0000", 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015839599999999999 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526121+0000", 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2785e-05 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530775+0000", 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046537870000000004 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530868+0000", 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3088e-05 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.1s2 33/31 ECSubWrite(tid=3, reqid=client.4568.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:01.434 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526871+0000", 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.8985985970000003, 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10188062000000001, 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526871+0000", 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526871+0000", 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526873+0000", 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526917+0000", 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4326999999999998e-05 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9663000000000003e-05 2024-04-18T23:11:01.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526981+0000", 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3915999999999999e-05 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.627401+0000", 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10041953200000001 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.627425+0000", 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4085000000000001e-05 2024-04-18T23:11:01.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628646+0000", 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012212010000000001 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628752+0000", 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010604599999999999 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 33/31 ECSubWrite(tid=4, reqid=client.4568.0:17, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.527041+0000", 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.8984285449999998, 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050319240000000001, 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527041+0000", 2024-04-18T23:11:01.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527041+0000", 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527042+0000", 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1060000000000001e-06 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527085+0000", 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2404000000000003e-05 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527109+0000", 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3941999999999999e-05 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527128+0000", 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9664999999999999e-05 2024-04-18T23:11:01.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527248+0000", 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000119852 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.527272+0000", 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3527999999999999e-05 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531877+0000", 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0046051540000000002 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.532073+0000", 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019627300000000001 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 33/31 ECSubWrite(tid=5, reqid=client.4568.0:20, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:01.443 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.528268+0000", 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.8972016929999995, 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037628990000000001, 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528268+0000", 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528268+0000", 2024-04-18T23:11:01.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528270+0000", 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.745e-06 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528311+0000", 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0880000000000002e-05 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528351+0000", 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0175000000000001e-05 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528379+0000", 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8041000000000001e-05 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528597+0000", 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021789699999999999 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528619+0000", 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2055000000000001e-05 2024-04-18T23:11:01.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531922+0000", 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00330299 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.532031+0000", 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109116 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 34/31 ECSubWrite(tid=0, reqid=client.4568.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.535528+0000", 2024-04-18T23:11:01.447 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.889942059, 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.089577750999999997, 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535528+0000", 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535528+0000", 2024-04-18T23:11:01.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535529+0000", 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.22e-06 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535833+0000", 2024-04-18T23:11:01.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00030396900000000002 2024-04-18T23:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535865+0000", 2024-04-18T23:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1563999999999998e-05 2024-04-18T23:11:01.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535881+0000", 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6039e-05 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.536062+0000", 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018088199999999999 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.536086+0000", 2024-04-18T23:11:01.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4465e-05 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625023+0000", 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.088936666999999997 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625106+0000", 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2945e-05 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.452 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.537968+0000", 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.8875013970000003, 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.087241641999999994, 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537968+0000", 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.453 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537968+0000", 2024-04-18T23:11:01.454 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.455 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537970+0000", 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.33e-06 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538023+0000", 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3340000000000001e-05 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538048+0000", 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4851999999999999e-05 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538070+0000", 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1658000000000002e-05 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538180+0000", 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110676 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538198+0000", 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7261999999999998e-05 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625116+0000", 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.086918529999999994 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625210+0000", 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3993999999999995e-05 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=8, reqid=client.4568.0:40, at_version=34'3, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.735888+0000", 2024-04-18T23:11:01.458 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6895819029999997, 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033728830000000001, 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.735888+0000", 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.735888+0000", 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.735890+0000", 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.64e-06 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736059+0000", 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016960200000000001 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.460 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736092+0000", 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2644000000000001e-05 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.736114+0000", 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2466000000000001e-05 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737777+0000", 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016631110000000001 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.461 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737792+0000", 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4089e-05 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.739187+0000", 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013959160000000001 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.739261+0000", 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3415000000000001e-05 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=9, reqid=client.4568.0:41, at_version=34'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.737558+0000", 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6879120920000004, 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.098370887000000004, 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737558+0000", 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737558+0000", 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737559+0000", 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737599+0000", 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0030000000000001e-05 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737634+0000", 2024-04-18T23:11:01.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4378000000000002e-05 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737656+0000", 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2478000000000001e-05 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.834675+0000", 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.097018668000000002 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.834702+0000", 2024-04-18T23:11:01.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7625000000000001e-05 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.835833+0000", 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011305740000000001 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.835929+0000", 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5626999999999998e-05 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=10, reqid=client.4568.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.825050+0000", 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.600419424, 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10149166599999999, 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825050+0000", 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825050+0000", 2024-04-18T23:11:01.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825052+0000", 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.61e-06 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825085+0000", 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2428000000000002e-05 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825112+0000", 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7168000000000001e-05 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825133+0000", 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1061999999999999e-05 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.925428+0000", 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100294913 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.925442+0000", 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4528999999999999e-05 2024-04-18T23:11:01.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.926453+0000", 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001010515 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.926542+0000", 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9441e-05 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.470 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.1s2 34/31 ECSubRead(tid=11, to_read={3:8af79e0a:::smithi14230322-1:head=0,4096,0}, subchunks={3:8af79e0a:::smithi14230322-1:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.825619+0000", 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.5998505590000001, 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10067092800000001, 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825619+0000", 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825619+0000", 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825622+0000", 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2900000000000001e-06 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825640+0000", 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8289000000000001e-05 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825681+0000", 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1066999999999998e-05 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825721+0000", 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0244000000000003e-05 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.925824+0000", 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010322000000001 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.926290+0000", 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000465818 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 34/31 ECSubWrite(tid=5, reqid=client.4568.0:49, at_version=0'0, trim_to=0'0, roll_forward_to=34'3))", 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:57.040527+0000", 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.3849424079999997, 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0039768870000000001, 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.040527+0000", 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.474 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.040527+0000", 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.040529+0000", 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.314e-06 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.040556+0000", 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7548000000000001e-05 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.040584+0000", 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7790999999999999e-05 2024-04-18T23:11:01.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.040606+0000", 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1396e-05 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.040855+0000", 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00024902399999999998 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.040879+0000", 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4358e-05 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.044404+0000", 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035251800000000002 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.044504+0000", 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000100276 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 34/31 ECSubWrite(tid=6, reqid=client.4568.0:50, at_version=34'4, trim_to=0'0, roll_forward_to=34'3))", 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:57.042450+0000", 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.383019397, 2024-04-18T23:11:01.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0045801110000000004, 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.042450+0000", 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.042450+0000", 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.042452+0000", 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.042491+0000", 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9317999999999999e-05 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.042527+0000", 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5607e-05 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.042545+0000", 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8465999999999999e-05 2024-04-18T23:11:01.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.042679+0000", 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013406099999999999 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.042694+0000", 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4936000000000001e-05 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.046961+0000", 2024-04-18T23:11:01.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.004266461 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.047031+0000", 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9882000000000006e-05 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 36/31 ECSubWrite(tid=54, reqid=client.4568.0:188, at_version=36'19, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.658481+0000", 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.76698879600000003, 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10146903, 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.481 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658481+0000", 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658481+0000", 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658481+0000", 2024-04-18T23:11:01.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.92e-07 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658577+0000", 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5395000000000004e-05 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658601+0000", 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4672e-05 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658610+0000", 2024-04-18T23:11:01.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8389999999999996e-06 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.758786+0000", 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100175606 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.758793+0000", 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2710000000000003e-06 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.759907+0000", 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001113916 2024-04-18T23:11:01.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.759950+0000", 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3139000000000002e-05 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 36/31 ECSubWrite(tid=55, reqid=client.4568.0:181, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.660823+0000", 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.76464687600000003, 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088911570000000002, 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.485 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.660823+0000", 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.660823+0000", 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.660824+0000", 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.47e-07 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661023+0000", 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019883800000000001 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661044+0000", 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1175999999999999e-05 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661058+0000", 2024-04-18T23:11:01.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3872e-05 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662186+0000", 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001128059 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662197+0000", 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1579e-05 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669659+0000", 2024-04-18T23:11:01.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074618050000000002 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669714+0000", 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4880999999999999e-05 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 36/31 ECSubWrite(tid=56, reqid=client.4568.0:182, at_version=36'14, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.661892+0000", 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.76357781700000005, 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078648880000000004, 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.489 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661892+0000", 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661892+0000", 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661893+0000", 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.54e-07 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662067+0000", 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017397800000000001 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662085+0000", 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8085e-05 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662103+0000", 2024-04-18T23:11:01.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7949e-05 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.663087+0000", 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098454300000000005 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.663098+0000", 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0443e-05 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669720+0000", 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066224420000000001 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669757+0000", 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6594000000000002e-05 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 36/31 ECSubWrite(tid=57, reqid=client.4568.0:183, at_version=36'15, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.662696+0000", 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.76277382000000005, 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073676760000000001, 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.493 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662696+0000", 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662696+0000", 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662697+0000", 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.217e-06 2024-04-18T23:11:01.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662718+0000", 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1205000000000001e-05 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662745+0000", 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6580999999999998e-05 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662759+0000", 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3900000000000001e-05 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669313+0000", 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065537579999999998 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669330+0000", 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7736000000000001e-05 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.670027+0000", 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069616800000000005 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.496 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.670064+0000", 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7110999999999998e-05 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.es1 36/31 ECSubRead(tid=15, to_read={3:78aee268:::smithi14230322-13:head=0,53248,0}, subchunks={3:78aee268:::smithi14230322-13:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.770553+0000", 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.65491668999999997, 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100377276, 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770553+0000", 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770553+0000", 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770555+0000", 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9740000000000001e-06 2024-04-18T23:11:01.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770577+0000", 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1455e-05 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770614+0000", 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7800999999999999e-05 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870693+0000", 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100078207 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870835+0000", 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014253700000000001 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870930+0000", 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5302000000000003e-05 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.es1 36/31 ECSubRead(tid=16, to_read={3:7a4badf6:::smithi14230322-17:head=0,69632,0}, subchunks={3:7a4badf6:::smithi14230322-17:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:01.500 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.870730+0000", 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.554739859, 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20070310299999999, 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870730+0000", 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870730+0000", 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870731+0000", 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4099999999999998e-07 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870743+0000", 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2034e-05 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870760+0000", 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6956000000000002e-05 2024-04-18T23:11:01.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870776+0000", 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6276000000000001e-05 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070936+0000", 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20016004100000001 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071433+0000", 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049705500000000004 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.503 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 36/31 ECSubWrite(tid=65, reqid=client.4568.0:212, at_version=0'0, trim_to=0'0, roll_forward_to=36'25))", 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:01.172511+0000", 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.25295932900000001, 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101623536, 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172511+0000", 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172511+0000", 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172513+0000", 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4329999999999998e-06 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172544+0000", 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1068000000000001e-05 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172578+0000", 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4155000000000001e-05 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172596+0000", 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7325000000000001e-05 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.272858+0000", 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10026249199999999 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:01.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.272887+0000", 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9306e-05 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.274061+0000", 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011737410000000001 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.274134+0000", 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3015999999999993e-05 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.507 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:01.508 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:01.425+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:01.508 INFO:tasks.rados.rados.0.smithi142.stdout:69: finishing copy_from to smithi14230322-7 2024-04-18T23:11:01.509 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 7 v 4 (ObjNum 16 snap 0 seq_num 16) dirty exists 2024-04-18T23:11:01.509 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:11:01.526 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:01.522+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:64: done (15 left) 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:65: done (14 left) 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:66: done (13 left) 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:67: done (12 left) 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:68: done (11 left) 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:69: done (10 left) 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:70: done (9 left) 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:71: done (8 left) 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:72: done (7 left) 2024-04-18T23:11:01.541 INFO:tasks.rados.rados.0.smithi142.stdout:73: done (6 left) 2024-04-18T23:11:01.542 INFO:tasks.rados.rados.0.smithi142.stdout:75: done (5 left) 2024-04-18T23:11:01.542 INFO:tasks.rados.rados.0.smithi142.stdout:76: done (4 left) 2024-04-18T23:11:01.542 INFO:tasks.rados.rados.0.smithi142.stdout:77: done (3 left) 2024-04-18T23:11:01.542 INFO:tasks.rados.rados.0.smithi142.stdout:78: done (2 left) 2024-04-18T23:11:01.542 INFO:tasks.rados.rados.0.smithi142.stdout:79: done (1 left) 2024-04-18T23:11:01.542 INFO:tasks.rados.rados.0.smithi142.stdout:80: done (0 left) 2024-04-18T23:11:01.542 INFO:tasks.rados.rados.0.smithi142.stdout:81: rollback oid 40 current snap is 2 2024-04-18T23:11:01.542 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 40 to 1 2024-04-18T23:11:01.542 INFO:tasks.rados.rados.0.smithi142.stdout:82: snap_create 2024-04-18T23:11:01.627 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:01.625+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:01.638 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:01.638 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:01.639 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:01.639 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:01.653 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:11:01.728 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:01.725+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:01.774 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:01.774 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:01.774 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:01.774 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:01.774 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:01.788 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:11:01.834 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:01.837+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:01.847 INFO:tasks.rados.rados.0.smithi142.stdout:81: finishing rollback tid 1 to smithi14230322-40 2024-04-18T23:11:01.848 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 40 v 0 (ObjNum 129 snap 0 seq_num 1439604170) dirty dne 2024-04-18T23:11:01.910 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:01.910 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.531716883000001, 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50897375199999995, 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:11:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:11:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377587+0000", 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3089999999999998e-06 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377601+0000", 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3862999999999999e-05 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377623+0000", 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1741000000000001e-05 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377644+0000", 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1024000000000001e-05 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.576350+0000", 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198706355 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.576365+0000", 2024-04-18T23:11:01.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4748e-05 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.682700+0000", 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106335283 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.683098+0000", 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039820900000000001 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.683616+0000", 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051731399999999995 2024-04-18T23:11:01.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.688337+0000", 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047211509999999998 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886392+0000", 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198055008 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886494+0000", 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102203 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886559+0000", 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4543999999999997e-05 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.315041896, 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41468469299999999, 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.917 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 110 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:11:01.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594261+0000", 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594266+0000", 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9690000000000002e-06 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594278+0000", 2024-04-18T23:11:01.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2048999999999999e-05 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594286+0000", 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2900000000000002e-06 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.804556+0000", 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.21026968800000001 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.804784+0000", 2024-04-18T23:11:01.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022826000000000001 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.805113+0000", 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032918500000000001 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.908604+0000", 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103490558 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008799+0000", 2024-04-18T23:11:01.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100195153 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008880+0000", 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1329999999999996e-05 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008945+0000", 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4400999999999995e-05 2024-04-18T23:11:01.922 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.315004126, 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41616193699999998, 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.923 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 111 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.924 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6300000000000004e-07 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594305+0000", 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594317+0000", 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1786e-05 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594327+0000", 2024-04-18T23:11:01.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7249999999999992e-06 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906203+0000", 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.31187656499999999 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906336+0000", 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013266800000000001 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906542+0000", 2024-04-18T23:11:01.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020588900000000001 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009954+0000", 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10341181100000001 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010185+0000", 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023115700000000001 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010324+0000", 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013910800000000001 2024-04-18T23:11:01.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010460+0000", 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000135635 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.314963392999999, 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42114173300000002, 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.928 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 112 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:11:01.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594339+0000", 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7700000000000004e-07 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594484+0000", 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000144804 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594500+0000", 2024-04-18T23:11:01.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5752000000000001e-05 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594511+0000", 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1039000000000001e-05 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907178+0000", 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.31266760199999999 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907320+0000", 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014133099999999999 2024-04-18T23:11:01.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907545+0000", 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022546 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010139+0000", 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102593936 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015192+0000", 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050530590000000004 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015364+0000", 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017223600000000001 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015480+0000", 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011573700000000001 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.933 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.314774595999999, 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42137598300000001, 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 113 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.934 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594528+0000", 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594533+0000", 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0130000000000001e-06 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594545+0000", 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2009e-05 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594553+0000", 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7389999999999999e-06 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009008+0000", 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41445484999999999 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009125+0000", 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117773 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009350+0000", 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022510400000000001 2024-04-18T23:11:01.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015618+0000", 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062671189999999998 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015662+0000", 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4619000000000001e-05 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015798+0000", 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013600600000000001 2024-04-18T23:11:01.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015903+0000", 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104941 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.314737276000001, 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62069844100000005, 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.939 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 114 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:01.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594565+0000", 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594570+0000", 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.186e-06 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594582+0000", 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1763000000000001e-05 2024-04-18T23:11:01.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594590+0000", 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5290000000000003e-06 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010572+0000", 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.415982252 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010727+0000", 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015558400000000001 2024-04-18T23:11:01.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010971+0000", 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000243339 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015929+0000", 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049583639999999998 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215108+0000", 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19917934400000001 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215199+0000", 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0556000000000005e-05 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215263+0000", 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3874000000000005e-05 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.944 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.314700949000001, 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62089894099999998, 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 115 2024-04-18T23:11:01.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:01.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594746+0000", 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014446700000000001 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594761+0000", 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4792e-05 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594769+0000", 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6160000000000002e-06 2024-04-18T23:11:01.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013742+0000", 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.418972438 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013974+0000", 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023265699999999999 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.014328+0000", 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035345 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018069+0000", 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037411079999999999 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215346+0000", 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19727757900000001 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215435+0000", 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8245999999999995e-05 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215500+0000", 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4968000000000005e-05 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.314517698, 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.72133309899999998, 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.950 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 116 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594785+0000", 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594790+0000", 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4229999999999996e-06 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594803+0000", 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2196e-05 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594810+0000", 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6190000000000002e-06 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016062+0000", 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42125235 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016215+0000", 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000152051 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016420+0000", 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000205916 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018082+0000", 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001661718 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.315968+0000", 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29788616600000001 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316051+0000", 2024-04-18T23:11:01.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2897000000000003e-05 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316117+0000", 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5866999999999998e-05 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:01.956 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 14.314480065, 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.721588585, 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 117 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:01.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594829+0000", 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7830000000000001e-06 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594841+0000", 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1505000000000001e-05 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.694940+0000", 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100099585 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016967+0000", 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32202692799999999 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017224+0000", 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000256548 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017583+0000", 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000359689 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.020139+0000", 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025556649999999999 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316256+0000", 2024-04-18T23:11:01.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29611688800000002 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316345+0000", 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9285999999999999e-05 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316410+0000", 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5058e-05 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.962 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.834678675999999, 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.410819936, 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.963 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 140 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:11:01.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074624+0000", 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074804+0000", 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018036800000000001 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074824+0000", 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0018e-05 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074839+0000", 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4846e-05 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080014+0000", 2024-04-18T23:11:01.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051746780000000003 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080068+0000", 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4345000000000002e-05 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080162+0000", 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4157e-05 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081144+0000", 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098186100000000011 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485220+0000", 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40407592199999998 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485314+0000", 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4161999999999997e-05 2024-04-18T23:11:01.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485443+0000", 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012892900000000001 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:11:01.969 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.834445112999999, 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41085222500000002, 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 141 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074858+0000", 2024-04-18T23:11:01.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0139999999999999e-06 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074863+0000", 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3759999999999999e-06 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074872+0000", 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3929999999999997e-06 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074883+0000", 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0722e-05 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080730+0000", 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058473249999999996 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080788+0000", 2024-04-18T23:11:01.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7423999999999998e-05 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080881+0000", 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2938999999999999e-05 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081458+0000", 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057760199999999998 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485586+0000", 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40412787700000002 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485656+0000", 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0034000000000001e-05 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485709+0000", 2024-04-18T23:11:01.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2519000000000003e-05 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.775219569000001, 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50946670199999999, 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.976 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 162 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:01.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134083+0000", 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7000000000000003e-07 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134088+0000", 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2719999999999997e-06 2024-04-18T23:11:01.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134097+0000", 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1400000000000006e-06 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134104+0000", 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140241+0000", 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0061371919999999996 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140319+0000", 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7768000000000005e-05 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140446+0000", 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126637 2024-04-18T23:11:01.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243778+0000", 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103331863 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643391+0000", 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.39961375599999999 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643476+0000", 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4274e-05 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643549+0000", 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3071000000000006e-05 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.775186409, 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50963481200000005, 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 163 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.983 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134116+0000", 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134123+0000", 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4930000000000003e-06 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134131+0000", 2024-04-18T23:11:01.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9490000000000002e-06 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134138+0000", 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8399999999999997e-06 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140848+0000", 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067093389999999999 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140969+0000", 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121787 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141164+0000", 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019482700000000001 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243782+0000", 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10261785900000001 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643626+0000", 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.399843858 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643699+0000", 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2657999999999995e-05 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643750+0000", 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1481999999999999e-05 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.775152318, 2024-04-18T23:11:01.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50988570200000005, 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 164 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134150+0000", 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4000000000000001e-07 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134332+0000", 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018177199999999999 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134351+0000", 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9321999999999999e-05 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134368+0000", 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6994999999999999e-05 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141740+0000", 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073716420000000003 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141823+0000", 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3042999999999998e-05 2024-04-18T23:11:01.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141951+0000", 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127976 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243787+0000", 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101836063 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643874+0000", 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40008676199999998 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643964+0000", 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0362e-05 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.644035+0000", 2024-04-18T23:11:01.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1124999999999994e-05 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.774893082, 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.54739116099999996, 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:01.996 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 165 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:01.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134410+0000", 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5770000000000001e-06 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134420+0000", 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0175e-05 2024-04-18T23:11:01.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134439+0000", 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8125e-05 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134453+0000", 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4956e-05 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:01.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242648+0000", 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.108194819 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242783+0000", 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013431599999999999 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242992+0000", 2024-04-18T23:11:02.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020903 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.244280+0000", 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012883669999999999 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681587+0000", 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.43730722300000002 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681676+0000", 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8776000000000005e-05 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681800+0000", 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012379700000000001 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.002 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:11 3.2s0 3:42341748:::smithi14230322-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3910030320000004, 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50900182400000005, 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 11 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.003 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:02.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518300+0000", 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.823e-06 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518324+0000", 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3238999999999999e-05 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518345+0000", 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0828000000000001e-05 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518359+0000", 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4912000000000001e-05 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521623+0000", 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003263808 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:02.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.522045+0000", 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00042140500000000002 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.523231+0000", 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011861840000000001 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530352+0000", 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071208340000000004 2024-04-18T23:11:02.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.027150+0000", 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49679807100000001 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.027300+0000", 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015071999999999999 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.008 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:25 3.2s0 3:4250b374:::smithi14230322-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3874231320000003, 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.51011498, 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:02.009 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 25 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521881+0000", 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4430000000000002e-06 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524055+0000", 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021740499999999999 2024-04-18T23:11:02.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524086+0000", 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1414000000000001e-05 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524107+0000", 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0716e-05 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:02.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524323+0000", 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215675 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524696+0000", 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037315500000000001 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526737+0000", 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020405380000000002 2024-04-18T23:11:02.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530355+0000", 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036182720000000001 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.031805+0000", 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50145000299999998 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.031994+0000", 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018871400000000001 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:26 3.2s0 3:4250b374:::smithi14230322-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3850940390000002, 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60852922899999995, 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:02.015 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 26 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524210+0000", 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524220+0000", 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0056e-05 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524242+0000", 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2212e-05 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525734+0000", 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001491783 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528099+0000", 2024-04-18T23:11:02.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023654380000000001 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528318+0000", 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000219065 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.529513+0000", 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011952530000000001 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:02.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636414+0000", 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106900906 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132457+0000", 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49604301899999997 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132737+0000", 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027950099999999998 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.020 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:31 3.2s0 3:42c34e84:::smithi14230322-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.382979615, 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60793644899999999, 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:02.021 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 31 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526324+0000", 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9400000000000001e-06 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526836+0000", 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051192200000000005 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526869+0000", 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2978000000000002e-05 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526890+0000", 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481999999999999e-05 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531167+0000", 2024-04-18T23:11:02.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042762490000000002 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531508+0000", 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034109400000000001 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.533620+0000", 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021122279999999998 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636418+0000", 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102798509 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134075+0000", 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.497656453 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134259+0000", 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000183594 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:32 3.2s0 3:42c34e84:::smithi14230322-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3823767050000004, 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60874687999999999, 2024-04-18T23:11:02.027 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 32 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526927+0000", 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0389999999999999e-06 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526936+0000", 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2599999999999994e-06 2024-04-18T23:11:02.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0302e-05 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526973+0000", 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6359e-05 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:02.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634473+0000", 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.107499583 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634639+0000", 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016687200000000001 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.635580+0000", 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094034999999999995 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636830+0000", 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012506220000000001 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135509+0000", 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49867835399999999 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:02.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135672+0000", 2024-04-18T23:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000163139 2024-04-18T23:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:02.033 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:02.035 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:11:02.035 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:02.035+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:02.136 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:02.137+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:02.165 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:02.165 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:11:02.165 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:02.179 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:11:02.237 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:02.235+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:02.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:02.303 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:02.303 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:02.304 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:02.317 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:11:02.338 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:02.334+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:02.397 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:02.397 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:02.417 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:02.418 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:02.436 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:02.436 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:02.439 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:02.441+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:02.457 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:02.457 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:02.473 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:02.473 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:02.484 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:02.484 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:02.494 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:02.494 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:02.498 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:11:02.515 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:02.515 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:02.536 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:02.536 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:81: done (1 left) 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:82: done (0 left) 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:83: rollback oid 9 current snap is 3 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 9 to 0 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:84: read oid 20 snap 1 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:84: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:85: read oid 41 snap -1 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:86: read oid 5 snap -1 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:86: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:87: read oid 4 snap -1 2024-04-18T23:11:02.546 INFO:tasks.rados.rados.0.smithi142.stdout:87: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-18T23:11:02.547 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 3 current snap is 3 2024-04-18T23:11:02.547 INFO:tasks.rados.rados.0.smithi142.stdout:88: seq_num 54 ranges {491520=540672,1032192=376832} 2024-04-18T23:11:02.547 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:02.547+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:02.553 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T23:11:02.555 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:02.555 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:02.562 INFO:tasks.rados.rados.0.smithi142.stdout:88: writing smithi14230322-3 from 491520 to 1032192 tid 1 2024-04-18T23:11:02.569 INFO:tasks.rados.rados.0.smithi142.stdout:88: writing smithi14230322-3 from 1032192 to 1409024 tid 2 2024-04-18T23:11:02.569 INFO:tasks.rados.rados.0.smithi142.stdout:89: read oid 27 snap -1 2024-04-18T23:11:02.569 INFO:tasks.rados.rados.0.smithi142.stdout:89: expect (ObjNum 26 snap 0 seq_num 26) 2024-04-18T23:11:02.569 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 17 current snap is 3 2024-04-18T23:11:02.569 INFO:tasks.rados.rados.0.smithi142.stdout:90: seq_num 55 ranges {278528=557056,835584=376832} 2024-04-18T23:11:02.577 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:02.577 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:02.582 INFO:tasks.rados.rados.0.smithi142.stdout:90: writing smithi14230322-17 from 278528 to 835584 tid 1 2024-04-18T23:11:02.589 INFO:tasks.rados.rados.0.smithi142.stdout:90: writing smithi14230322-17 from 835584 to 1212416 tid 2 2024-04-18T23:11:02.590 INFO:tasks.rados.rados.0.smithi142.stdout:91: rollback oid 36 current snap is 3 2024-04-18T23:11:02.590 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 36 to 2 2024-04-18T23:11:02.590 INFO:tasks.rados.rados.0.smithi142.stdout:92: rollback oid 7 current snap is 3 2024-04-18T23:11:02.590 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 7 to 0 2024-04-18T23:11:02.590 INFO:tasks.rados.rados.0.smithi142.stdout:93: snap_create 2024-04-18T23:11:02.594 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:02.594 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:02.615 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:02.615 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:02.624 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:02.624 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:02.638 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:11:02.641 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:02.641+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:02.665 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:02.665 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:02.694 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:02.694 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:02.712 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:02.712 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:02.733 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:02.733 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:02.741 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:02.738+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:02.809 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:02.809 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:02.823 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:11:02.842 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:02.845+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:02.857 INFO:tasks.rados.rados.0.smithi142.stdout:83: finishing rollback tid 0 to smithi14230322-9 2024-04-18T23:11:02.858 INFO:tasks.rados.rados.0.smithi142.stdout:86: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-18T23:11:02.898 INFO:tasks.rados.rados.0.smithi142.stdout:91: finishing rollback tid 0 to smithi14230322-36 2024-04-18T23:11:02.943 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:02.945+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:02.948 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:02.948 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:02.952 INFO:tasks.rados.rados.0.smithi142.stdout:84: expect (ObjNum 19 snap 0 seq_num 19) 2024-04-18T23:11:02.963 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:11:03.043 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:03.038+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:03.069 INFO:tasks.rados.rados.0.smithi142.stdout:88: finishing write tid 1 to smithi14230322-3 2024-04-18T23:11:03.086 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:03.086 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:03.099 INFO:tasks.rados.rados.0.smithi142.stdout:92: finishing rollback tid 0 to smithi14230322-7 2024-04-18T23:11:03.099 INFO:tasks.rados.rados.0.smithi142.stdout:91: finishing rollback tid 1 to smithi14230322-36 2024-04-18T23:11:03.101 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:11:03.101 INFO:tasks.rados.rados.0.smithi142.stdout:85: expect (ObjNum 40 snap 0 seq_num 40) 2024-04-18T23:11:03.144 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:03.145+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:03.200 INFO:tasks.rados.rados.0.smithi142.stdout:91: finishing rollback tid 2 to smithi14230322-36 2024-04-18T23:11:03.200 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 36 v 29 (ObjNum 35 snap 0 seq_num 35) dirty exists 2024-04-18T23:11:03.200 INFO:tasks.rados.rados.0.smithi142.stdout:92: finishing rollback tid 1 to smithi14230322-7 2024-04-18T23:11:03.219 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:03.219 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:03.234 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:11:03.245 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:03.242+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:03.293 INFO:tasks.rados.rados.0.smithi142.stdout:90: finishing write tid 1 to smithi14230322-17 2024-04-18T23:11:03.293 INFO:tasks.rados.rados.0.smithi142.stdout:90: finishing write tid 2 to smithi14230322-17 2024-04-18T23:11:03.293 INFO:tasks.rados.rados.0.smithi142.stdout:90: finishing write tid 3 to smithi14230322-17 2024-04-18T23:11:03.294 INFO:tasks.rados.rados.0.smithi142.stdout:87: expect (ObjNum 3 snap 0 seq_num 3) 2024-04-18T23:11:03.300 INFO:tasks.rados.rados.0.smithi142.stdout:83: finishing rollback tid 1 to smithi14230322-9 2024-04-18T23:11:03.300 INFO:tasks.rados.rados.0.smithi142.stdout:83: finishing rollback tid 2 to smithi14230322-9 2024-04-18T23:11:03.300 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 9 v 12 (ObjNum 8 snap 0 seq_num 8) dirty exists 2024-04-18T23:11:03.346 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:03.349+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:03.357 INFO:tasks.rados.rados.0.smithi142.stdout:92: finishing rollback tid 2 to smithi14230322-7 2024-04-18T23:11:03.357 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 7 v 21 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-18T23:11:03.358 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:03.358 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:03.372 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:11:03.447 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:03.444+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:11:03.495 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:03.495 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:03.496 INFO:tasks.rados.rados.0.smithi142.stdout:90: finishing write tid 4 to smithi14230322-17 2024-04-18T23:11:03.496 INFO:tasks.rados.rados.0.smithi142.stdout:90: oid 17 updating version 0 to 8 2024-04-18T23:11:03.496 INFO:tasks.rados.rados.0.smithi142.stdout:90: oid 17 updating version 8 to 10 2024-04-18T23:11:03.496 INFO:tasks.rados.rados.0.smithi142.stdout:90: oid 17 version 10 is already newer than 9 2024-04-18T23:11:03.496 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 17 v 10 (ObjNum 55 snap 3 seq_num 55) dirty exists 2024-04-18T23:11:03.496 INFO:tasks.rados.rados.0.smithi142.stdout:90: left oid 17 (ObjNum 55 snap 3 seq_num 55) 2024-04-18T23:11:03.509 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:11:03.547 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:03.548+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:83: done (10 left) 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:84: done (9 left) 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:85: done (8 left) 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:86: done (7 left) 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:87: done (6 left) 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:90: done (5 left) 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:91: done (4 left) 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:92: done (3 left) 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:93: done (2 left) 2024-04-18T23:11:03.553 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 2 current snap is 4 2024-04-18T23:11:03.554 INFO:tasks.rados.rados.0.smithi142.stdout:94: seq_num 56 ranges {180224=573440,753664=376832} 2024-04-18T23:11:03.555 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-04-18T23:11:03.564 INFO:tasks.rados.rados.0.smithi142.stdout:94: writing smithi14230322-2 from 180224 to 753664 tid 1 2024-04-18T23:11:03.570 INFO:tasks.rados.rados.0.smithi142.stdout:94: writing smithi14230322-2 from 753664 to 1130496 tid 2 2024-04-18T23:11:03.570 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 16 current snap is 4 2024-04-18T23:11:03.570 INFO:tasks.rados.rados.0.smithi142.stdout:95: seq_num 57 ranges {262144=589824,851968=376832} 2024-04-18T23:11:03.584 INFO:tasks.rados.rados.0.smithi142.stdout:95: writing smithi14230322-16 from 262144 to 851968 tid 1 2024-04-18T23:11:03.590 INFO:tasks.rados.rados.0.smithi142.stdout:95: writing smithi14230322-16 from 851968 to 1228800 tid 2 2024-04-18T23:11:03.590 INFO:tasks.rados.rados.0.smithi142.stdout:96: read oid 28 snap -1 2024-04-18T23:11:03.590 INFO:tasks.rados.rados.0.smithi142.stdout:96: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-18T23:11:03.590 INFO:tasks.rados.rados.0.smithi142.stdout:97: rollback oid 33 current snap is 4 2024-04-18T23:11:03.590 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 33 to 3 2024-04-18T23:11:03.590 INFO:tasks.rados.rados.0.smithi142.stdout:88: finishing write tid 2 to smithi14230322-3 2024-04-18T23:11:03.590 INFO:tasks.rados.rados.0.smithi142.stdout:88: finishing write tid 3 to smithi14230322-3 2024-04-18T23:11:03.590 INFO:tasks.rados.rados.0.smithi142.stdout:89: expect (ObjNum 26 snap 0 seq_num 26) 2024-04-18T23:11:03.622 INFO:tasks.rados.rados.0.smithi142.stdout:89: done (5 left) 2024-04-18T23:11:03.622 INFO:tasks.rados.rados.0.smithi142.stdout:98: snap_remove snap 0 2024-04-18T23:11:03.627 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:03.628 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:03.642 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:11:03.648 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:03.646+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:03.749 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:03.744+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:03.765 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:03.765 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:03.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:11:03.849 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:03.846+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:03.911 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:03.911 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:03.926 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:11:03.950 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:03.946+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:04.049 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:04.049 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:04.051 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:04.048+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:04.064 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:11:04.152 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:04.150+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:04.228 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:04.228 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:04.242 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:11:04.289 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:04.288+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:11:04.353 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:04.352+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:04.363 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:04.363 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:04.377 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:11:04.454 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:04.450+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:04.547 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:04.547 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:04.555 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:04.555+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:04.561 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:11:04.562 INFO:tasks.rados.rados.0.smithi142.stdout:98: done (5 left) 2024-04-18T23:11:04.562 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 12 current snap is 4 2024-04-18T23:11:04.562 INFO:tasks.rados.rados.0.smithi142.stdout:88: finishing write tid 4 to smithi14230322-3 2024-04-18T23:11:04.562 INFO:tasks.rados.rados.0.smithi142.stdout:88: oid 3 updating version 0 to 21 2024-04-18T23:11:04.562 INFO:tasks.rados.rados.0.smithi142.stdout:88: oid 3 version 21 is already newer than 20 2024-04-18T23:11:04.562 INFO:tasks.rados.rados.0.smithi142.stdout:88: oid 3 version 21 is already newer than 19 2024-04-18T23:11:04.562 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 3 v 21 (ObjNum 54 snap 3 seq_num 54) dirty exists 2024-04-18T23:11:04.563 INFO:tasks.rados.rados.0.smithi142.stdout:88: left oid 3 (ObjNum 54 snap 3 seq_num 54) 2024-04-18T23:11:04.563 INFO:tasks.rados.rados.0.smithi142.stdout:94: finishing write tid 1 to smithi14230322-2 2024-04-18T23:11:04.563 INFO:tasks.rados.rados.0.smithi142.stdout:94: finishing write tid 2 to smithi14230322-2 2024-04-18T23:11:04.563 INFO:tasks.rados.rados.0.smithi142.stdout:94: finishing write tid 3 to smithi14230322-2 2024-04-18T23:11:04.563 INFO:tasks.rados.rados.0.smithi142.stdout:99: seq_num 58 ranges {196608=606208,802816=376832} 2024-04-18T23:11:04.569 INFO:tasks.rados.rados.0.smithi142.stdout:99: writing smithi14230322-12 from 196608 to 802816 tid 1 2024-04-18T23:11:04.572 INFO:tasks.rados.rados.0.smithi142.stdout:99: writing smithi14230322-12 from 802816 to 1179648 tid 2 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:88: done (5 left) 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:100: read oid 15 snap -1 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:100: expect (ObjNum 10 snap 0 seq_num 10) 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:101: snap_create 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:95: finishing write tid 1 to smithi14230322-16 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:95: finishing write tid 2 to smithi14230322-16 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:95: finishing write tid 3 to smithi14230322-16 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:94: finishing write tid 4 to smithi14230322-2 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:94: oid 2 updating version 0 to 14 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:94: oid 2 version 14 is already newer than 13 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:94: oid 2 updating version 14 to 15 2024-04-18T23:11:04.573 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 2 v 15 (ObjNum 56 snap 4 seq_num 56) dirty exists 2024-04-18T23:11:04.574 INFO:tasks.rados.rados.0.smithi142.stdout:94: left oid 2 (ObjNum 56 snap 4 seq_num 56) 2024-04-18T23:11:04.574 INFO:tasks.rados.rados.0.smithi142.stdout:95: finishing write tid 4 to smithi14230322-16 2024-04-18T23:11:04.574 INFO:tasks.rados.rados.0.smithi142.stdout:95: oid 16 updating version 0 to 8 2024-04-18T23:11:04.574 INFO:tasks.rados.rados.0.smithi142.stdout:95: oid 16 version 8 is already newer than 7 2024-04-18T23:11:04.574 INFO:tasks.rados.rados.0.smithi142.stdout:95: oid 16 version 8 is already newer than 6 2024-04-18T23:11:04.575 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 16 v 8 (ObjNum 57 snap 4 seq_num 57) dirty exists 2024-04-18T23:11:04.575 INFO:tasks.rados.rados.0.smithi142.stdout:95: left oid 16 (ObjNum 57 snap 4 seq_num 57) 2024-04-18T23:11:04.575 INFO:tasks.rados.rados.0.smithi142.stdout:96: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-18T23:11:04.591 INFO:tasks.rados.rados.0.smithi142.stdout:97: finishing rollback tid 0 to smithi14230322-33 2024-04-18T23:11:04.591 INFO:tasks.rados.rados.0.smithi142.stdout:97: finishing rollback tid 1 to smithi14230322-33 2024-04-18T23:11:04.591 INFO:tasks.rados.rados.0.smithi142.stdout:97: finishing rollback tid 2 to smithi14230322-33 2024-04-18T23:11:04.591 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 33 v 13 (ObjNum 32 snap 0 seq_num 32) dirty exists 2024-04-18T23:11:04.656 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:04.654+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:04.692 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:04.692 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:04.708 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:11:04.751 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:04.756 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:04.755+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:04.857 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:04.854+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:04.872 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:04.872 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:04.881 INFO:tasks.rados.rados.0.smithi142.stdout:99: finishing write tid 1 to smithi14230322-12 2024-04-18T23:11:04.881 INFO:tasks.rados.rados.0.smithi142.stdout:99: finishing write tid 2 to smithi14230322-12 2024-04-18T23:11:04.887 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:11:04.908 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:04.908 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:04.929 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:04.929 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:04.958 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:04.955+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:04.981 INFO:tasks.rados.rados.0.smithi142.stdout:99: finishing write tid 3 to smithi14230322-12 2024-04-18T23:11:05.000 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:05.000 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:05.007 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:05.007 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:05.021 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:11:05.025 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:05.025 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:05.058 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:05.060+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:05.081 INFO:tasks.rados.rados.0.smithi142.stdout:100: expect (ObjNum 10 snap 0 seq_num 10) 2024-04-18T23:11:05.105 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:05.105 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:05.144 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:05.144 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:05.158 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:11:05.159 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:05.154+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:05.218 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:05.218 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:05.247 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:05.247 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:05.260 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:05.256+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:05.270 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:05.270 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:05.275 INFO:tasks.rados.rados.0.smithi142.stdout:99: finishing write tid 4 to smithi14230322-12 2024-04-18T23:11:05.275 INFO:tasks.rados.rados.0.smithi142.stdout:99: oid 12 updating version 0 to 8 2024-04-18T23:11:05.275 INFO:tasks.rados.rados.0.smithi142.stdout:99: oid 12 version 8 is already newer than 7 2024-04-18T23:11:05.275 INFO:tasks.rados.rados.0.smithi142.stdout:99: oid 12 version 8 is already newer than 6 2024-04-18T23:11:05.275 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 12 v 8 (ObjNum 58 snap 4 seq_num 58) dirty exists 2024-04-18T23:11:05.275 INFO:tasks.rados.rados.0.smithi142.stdout:99: left oid 12 (ObjNum 58 snap 4 seq_num 58) 2024-04-18T23:11:05.282 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:05.282 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:05.288 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:05.296 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-18T23:11:05.309 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:05.310 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:05.334 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:05.334 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:05.356 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:05.356 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:05.376 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:05.377 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:05.381 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:05.380+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:05.400 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:05.400 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:05.423 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:05.423 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:05.444 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:05.444 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:05.455 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:05.455 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:05.455 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:05.455 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:05.455 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:05.462 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:05.458+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:05.469 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-18T23:11:05.563 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:05.564+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:05.568 INFO:tasks.rados.rados.0.smithi142.stdout:94: done (6 left) 2024-04-18T23:11:05.568 INFO:tasks.rados.rados.0.smithi142.stdout:95: done (5 left) 2024-04-18T23:11:05.568 INFO:tasks.rados.rados.0.smithi142.stdout:96: done (4 left) 2024-04-18T23:11:05.568 INFO:tasks.rados.rados.0.smithi142.stdout:97: done (3 left) 2024-04-18T23:11:05.568 INFO:tasks.rados.rados.0.smithi142.stdout:99: done (2 left) 2024-04-18T23:11:05.568 INFO:tasks.rados.rados.0.smithi142.stdout:100: done (1 left) 2024-04-18T23:11:05.568 INFO:tasks.rados.rados.0.smithi142.stdout:101: done (0 left) 2024-04-18T23:11:05.569 INFO:tasks.rados.rados.0.smithi142.stdout:102: snap_remove snap 1 2024-04-18T23:11:05.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=1, reqid=client.4568.0:8, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.530503+0000", 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.1061789819999994, 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100798818, 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.638 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.530503+0000", 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.530503+0000", 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.530505+0000", 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3769999999999999e-06 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.530531+0000", 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6665e-05 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.530547+0000", 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5996000000000001e-05 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.530559+0000", 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1661999999999999e-05 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.630712+0000", 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100152807 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.630719+0000", 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4510000000000003e-06 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.631270+0000", 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00055116099999999999 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.631302+0000", 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1699000000000003e-05 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 33/31 ECSubWrite(tid=158, reqid=client.4568.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.622076+0000", 2024-04-18T23:11:05.642 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.0146062340000004, 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10109072199999999, 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622076+0000", 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622076+0000", 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622077+0000", 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7299999999999996e-07 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622157+0000", 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0477999999999998e-05 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622173+0000", 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5846e-05 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622179+0000", 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0399999999999998e-06 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622264+0000", 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5320000000000003e-05 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622276+0000", 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1834000000000001e-05 2024-04-18T23:11:05.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723083+0000", 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100807299 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723167+0000", 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3331999999999994e-05 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 33/31 ECSubWrite(tid=159, reqid=client.4568.0:25, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.622215+0000", 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.0144676990000008, 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10102508, 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622215+0000", 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622215+0000", 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622215+0000", 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3099999999999998e-07 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622276+0000", 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0630000000000001e-05 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622285+0000", 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.251e-06 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622303+0000", 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8054999999999999e-05 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622549+0000", 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00024611899999999999 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622559+0000", 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6609999999999995e-06 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723176+0000", 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10061779899999999 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723240+0000", 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3133999999999996e-05 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.650 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 33/31 ECSubWrite(tid=160, reqid=client.4568.0:26, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.622316+0000", 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.0143663719999996, 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10098744599999999, 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622316+0000", 2024-04-18T23:11:05.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622316+0000", 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622316+0000", 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1100000000000002e-07 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622334+0000", 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7643e-05 2024-04-18T23:11:05.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622341+0000", 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7870000000000004e-06 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622345+0000", 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3449999999999999e-06 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622979+0000", 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00063443600000000005 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622992+0000", 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2224e-05 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723247+0000", 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100255343 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723303+0000", 2024-04-18T23:11:05.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6357000000000003e-05 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 34/31 ECSubWrite(tid=161, reqid=client.4568.0:31, at_version=34'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.622375+0000", 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.0143069029999996, 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100987254, 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622375+0000", 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622375+0000", 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622376+0000", 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6299999999999999e-07 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622457+0000", 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1346000000000003e-05 2024-04-18T23:11:05.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622464+0000", 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9310000000000002e-06 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.622469+0000", 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8069999999999999e-06 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.630953+0000", 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0084844320000000001 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.630962+0000", 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0620000000000007e-06 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723308+0000", 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.092345981999999993 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723363+0000", 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4530999999999997e-05 2024-04-18T23:11:05.658 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 34/31 ECSubWrite(tid=162, reqid=client.4568.0:32, at_version=34'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.634781+0000", 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.0019008829999994, 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.089163563000000001, 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634781+0000", 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634781+0000", 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634782+0000", 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5899999999999996e-07 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634797+0000", 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5537999999999999e-05 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634806+0000", 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8009999999999993e-06 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634812+0000", 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2049999999999999e-06 2024-04-18T23:11:05.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.722531+0000", 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.087718638000000002 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.722560+0000", 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8422e-05 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723869+0000", 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013092609999999999 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723945+0000", 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6138999999999997e-05 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.663 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.0s2 34/31 ECSubWrite(tid=2, reqid=client.4568.0:46, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.734116+0000", 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.9025664100000004, 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100698999, 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734116+0000", 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.664 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734116+0000", 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734117+0000", 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0999999999999998e-07 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734342+0000", 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022516600000000001 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734357+0000", 2024-04-18T23:11:05.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5571000000000001e-05 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734367+0000", 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3889999999999994e-06 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734845+0000", 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00047792199999999998 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734852+0000", 2024-04-18T23:11:05.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0940000000000002e-06 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.834764+0000", 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099912123000000005 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.834815+0000", 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1023999999999998e-05 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.0s2 34/31 ECSubWrite(tid=3, reqid=client.4568.0:47, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.734711+0000", 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.9019717319999998, 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10049944, 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734711+0000", 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734711+0000", 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734711+0000", 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8299999999999996e-07 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734727+0000", 2024-04-18T23:11:05.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5821999999999999e-05 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734747+0000", 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9830000000000002e-05 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.734759+0000", 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1749999999999999e-05 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.834545+0000", 2024-04-18T23:11:05.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099786260000000002 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.834553+0000", 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1470000000000001e-06 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.835116+0000", 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00056302999999999995 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.835210+0000", 2024-04-18T23:11:05.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3818000000000002e-05 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 34/31 ECSubWrite(tid=165, reqid=client.4568.0:32, at_version=0'0, trim_to=0'0, roll_forward_to=34'6))", 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.135204+0000", 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.5014778300000007, 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10158044500000001, 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.672 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.135204+0000", 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.135204+0000", 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.135205+0000", 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3099999999999996e-07 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.135216+0000", 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1099e-05 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.135228+0000", 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1374e-05 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.135240+0000", 2024-04-18T23:11:05.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.257e-05 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.235524+0000", 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100283711 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.235558+0000", 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4017999999999999e-05 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.236742+0000", 2024-04-18T23:11:05.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011838110000000001 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.236785+0000", 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2930999999999999e-05 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=7, reqid=client.4568.0:64, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.141597+0000", 2024-04-18T23:11:05.676 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.4950855839999999, 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101626114, 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.141597+0000", 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.141597+0000", 2024-04-18T23:11:05.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.141597+0000", 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9599999999999999e-07 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.141849+0000", 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025182299999999998 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.141863+0000", 2024-04-18T23:11:05.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.429e-05 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.241963+0000", 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10009933999999999 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.242049+0000", 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6142000000000003e-05 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.242056+0000", 2024-04-18T23:11:05.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1169999999999998e-06 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.243197+0000", 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011404329999999999 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.243223+0000", 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6273000000000001e-05 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:73 3.5s0 3:a09a6894:::smithi14230322-25:head [append 0~409600,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.250638+0000", 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.3860447399999991, 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19068153099999999, 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 73 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.681 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.250638+0000", 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.250638+0000", 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.250640+0000", 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.283e-06 2024-04-18T23:11:05.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251228+0000", 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00058780699999999998 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251253+0000", 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5202e-05 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251271+0000", 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7938999999999999e-05 2024-04-18T23:11:05.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251342+0000", 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0902999999999995e-05 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251459+0000", 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011715299999999999 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.252406+0000", 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00094705400000000002 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.684 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.352179+0000", 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099773065999999994 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.441253+0000", 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.089074038999999994 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.441319+0000", 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6085000000000002e-05 2024-04-18T23:11:05.685 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:74 3.5s0 3:a09a6894:::smithi14230322-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.251310+0000", 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.3853724439999997, 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29140727700000002, 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:05.686 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 74 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251310+0000", 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251310+0000", 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.687 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251311+0000", 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2160000000000001e-06 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251322+0000", 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0946e-05 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251335+0000", 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2826000000000001e-05 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.688 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251345+0000", 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7499999999999998e-06 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.351471+0000", 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10012691 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.351554+0000", 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2670000000000006e-05 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.351932+0000", 2024-04-18T23:11:05.689 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00037827 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.352496+0000", 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00056402799999999995 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.542650+0000", 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19015369900000001 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.542717+0000", 2024-04-18T23:11:05.690 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6962e-05 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.254174+0000", 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.3825080360000008, 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098035816999999997, 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.691 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254174+0000", 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254174+0000", 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254176+0000", 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4759999999999999e-06 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.692 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254186+0000", 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8169999999999994e-06 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254212+0000", 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6310999999999999e-05 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.693 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254228+0000", 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5837000000000001e-05 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.352199+0000", 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.097970878999999997 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.352210+0000", 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1497e-05 2024-04-18T23:11:05.694 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.254401+0000", 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.3822815150000007, 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10032853, 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.695 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254401+0000", 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254401+0000", 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254402+0000", 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4499999999999996e-07 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.696 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254405+0000", 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1659999999999998e-06 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254415+0000", 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.004e-05 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.354500+0000", 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100084862 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.697 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.354718+0000", 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000218555 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.354729+0000", 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1062e-05 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:05.698 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.254954+0000", 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.3817279629999994, 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.097280723, 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254954+0000", 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254954+0000", 2024-04-18T23:11:05.699 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254955+0000", 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1260000000000001e-06 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254965+0000", 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0023e-05 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254984+0000", 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8238999999999999e-05 2024-04-18T23:11:05.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.254995+0000", 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.098e-05 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.352226+0000", 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.097231514000000005 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.701 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.352235+0000", 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8410000000000006e-06 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.6s3 34/31 ECSubWrite(tid=2, reqid=client.4568.0:35, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.339964+0000", 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.2967180749999994, 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10090110400000001, 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.702 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.339964+0000", 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.339964+0000", 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.339966+0000", 2024-04-18T23:11:05.703 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5290000000000001e-06 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.339983+0000", 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7096999999999999e-05 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.340006+0000", 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2753000000000001e-05 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.704 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.440085+0000", 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100079931 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.440177+0000", 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1534999999999997e-05 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.440202+0000", 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4674000000000001e-05 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.705 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.440834+0000", 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00063241399999999998 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.440865+0000", 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1170999999999997e-05 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 34/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.356242+0000", 2024-04-18T23:11:05.706 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.2804405620000008, 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.084190108, 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.356242+0000", 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.356242+0000", 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.356243+0000", 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5200000000000001e-06 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.356252+0000", 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5660000000000007e-06 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.356279+0000", 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6825000000000002e-05 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.356293+0000", 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3939e-05 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.440407+0000", 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.084114217000000005 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.440432+0000", 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5041e-05 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.709 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 34/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.442355+0000", 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.1943273140000006, 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10021633100000001, 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.710 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.442355+0000", 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.442355+0000", 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.442356+0000", 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0699999999999996e-07 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.711 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.442361+0000", 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3120000000000001e-06 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.442377+0000", 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6356999999999999e-05 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.542464+0000", 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100086841 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.542540+0000", 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6025999999999993e-05 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.542571+0000", 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0988000000000002e-05 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s4 34/31 ECSubWrite(tid=13, reqid=client.4568.0:76, at_version=34'5, trim_to=0'0, roll_forward_to=34'4))", 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.451422+0000", 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.1852602329999993, 2024-04-18T23:11:05.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10137368200000001, 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.451422+0000", 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.451422+0000", 2024-04-18T23:11:05.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.451422+0000", 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8400000000000005e-07 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.451562+0000", 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013970200000000001 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.451575+0000", 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3226e-05 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.551647+0000", 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10007176600000001 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.551746+0000", 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8778999999999997e-05 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.551755+0000", 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5690000000000005e-06 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.552754+0000", 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00099935299999999992 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.552796+0000", 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1802999999999999e-05 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.717 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 34/31 ECSubWriteReply(tid=2, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.544504+0000", 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.0921782429999993, 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10045641199999999, 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.544504+0000", 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.544504+0000", 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.544505+0000", 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.04e-06 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:05.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.544511+0000", 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5400000000000003e-06 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.544530+0000", 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8992999999999999e-05 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.644604+0000", 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100074553 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:05.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.644931+0000", 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032650700000000002 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.644960+0000", 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9779000000000001e-05 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:05.721 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:05.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:11:05.723 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:05.723+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:05.823 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:05.824+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:05.839 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:05.839 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:11:05.839 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:05.853 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:11:05.924 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:05.923+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:05.970 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:05.970 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:05.971 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:05.971 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:05.984 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:11:06.025 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:06.027+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:06.108 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:06.108 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 33/31 ECSubWrite(tid=1, reqid=client.4568.0:16, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.5882152709999993, 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104807236, 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6599999999999996e-07 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517909+0000", 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013603000000000001 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517960+0000", 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1122000000000003e-05 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.618057+0000", 2024-04-18T23:11:06.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10009630999999999 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.618206+0000", 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000149259 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.618219+0000", 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2761e-05 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.622544+0000", 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043254590000000002 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.622580+0000", 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5729000000000003e-05 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.8s1 34/31 ECSubWrite(tid=0, reqid=client.4568.0:43, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:06.113 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.619127+0000", 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.4868611719999993, 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.017069193, 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.619127+0000", 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.619127+0000", 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.619128+0000", 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0699999999999996e-07 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.619371+0000", 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024341400000000001 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.619394+0000", 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2855000000000001e-05 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.619408+0000", 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4017e-05 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.634363+0000", 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.014954861 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.634374+0000", 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0627e-05 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.636166+0000", 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001792746 2024-04-18T23:11:06.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.636196+0000", 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9765999999999999e-05 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.4s4 34/31 ECSubWrite(tid=2, reqid=client.4568.0:22, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:06.118 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.625577+0000", 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.4804112059999994, 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076011270000000001, 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625577+0000", 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625577+0000", 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625577+0000", 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.15e-07 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625622+0000", 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4487999999999998e-05 2024-04-18T23:11:06.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625632+0000", 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0369e-05 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625640+0000", 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5109999999999997e-06 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625836+0000", 2024-04-18T23:11:06.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000196837 2024-04-18T23:11:06.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625845+0000", 2024-04-18T23:11:06.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3739999999999998e-06 2024-04-18T23:11:06.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.633104+0000", 2024-04-18T23:11:06.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0072589439999999998 2024-04-18T23:11:06.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.633178+0000", 2024-04-18T23:11:06.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4189000000000004e-05 2024-04-18T23:11:06.123 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.123 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.cs2 36/31 ECSubRead(tid=15, to_read={3:35606aaf:::smithi14230322-20:head=0,81920,0}, subchunks={3:35606aaf:::smithi14230322-20:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.662623+0000", 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.4433654819999999, 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.060056532000000003, 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.662623+0000", 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.662623+0000", 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.662623+0000", 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7000000000000004e-07 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.662631+0000", 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0739999999999993e-06 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.662648+0000", 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6605000000000001e-05 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.662660+0000", 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1943999999999999e-05 2024-04-18T23:11:06.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.722538+0000", 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.059877970000000003 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.722679+0000", 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014116899999999999 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.127 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds4 36/31 ECSubWrite(tid=27, reqid=client.4568.0:187, at_version=36'15, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.749668+0000", 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.3563196729999998, 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10088845, 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.749668+0000", 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.749668+0000", 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.749669+0000", 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8199999999999999e-07 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.749684+0000", 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4994000000000001e-05 2024-04-18T23:11:06.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.749697+0000", 2024-04-18T23:11:06.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2700999999999999e-05 2024-04-18T23:11:06.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.749704+0000", 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.114e-06 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.849840+0000", 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100136218 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.849847+0000", 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2840000000000001e-06 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.850510+0000", 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00066229699999999995 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.850557+0000", 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7160000000000002e-05 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs1 36/31 ECSubWrite(tid=56, reqid=client.4568.0:182, at_version=36'14, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.759391+0000", 2024-04-18T23:11:06.132 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.3465974190000001, 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067646260000000001, 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759391+0000", 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759391+0000", 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759391+0000", 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8599999999999999e-07 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759436+0000", 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4759000000000003e-05 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759450+0000", 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4511e-05 2024-04-18T23:11:06.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759457+0000", 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4840000000000001e-06 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760250+0000", 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079310399999999999 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760259+0000", 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7539999999999995e-06 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766106+0000", 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0058475150000000002 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766155+0000", 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9113000000000002e-05 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.bs1 36/31 ECSubRead(tid=64, to_read={3:dd01ba5f:::smithi14230322-23:head=0,4096,0}, subchunks={3:dd01ba5f:::smithi14230322-23:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:06.136 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.068020+0000", 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.0379683310000001, 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100253277, 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068020+0000", 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068020+0000", 2024-04-18T23:11:06.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068020+0000", 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8000000000000001e-07 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068029+0000", 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8340000000000005e-06 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068043+0000", 2024-04-18T23:11:06.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4217e-05 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068058+0000", 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5119e-05 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168169+0000", 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100110607 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.139 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168273+0000", 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010412 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 36/31 ECSubWrite(tid=65, reqid=client.4568.0:212, at_version=0'0, trim_to=0'0, roll_forward_to=36'25))", 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.170058+0000", 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9359295110000003, 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101419938, 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170058+0000", 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170058+0000", 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170059+0000", 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7400000000000002e-07 2024-04-18T23:11:06.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170069+0000", 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0135e-05 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170082+0000", 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2186e-05 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170089+0000", 2024-04-18T23:11:06.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9650000000000002e-06 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.270270+0000", 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100181468 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.270287+0000", 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7110000000000001e-05 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.271425+0000", 2024-04-18T23:11:06.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001137281 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.271478+0000", 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3919000000000003e-05 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "pg_update_log_missing(3.9s2 epoch 37/31 rep_tid 41 entries 37'17 (0'0) error 3:9e84ac80:::smithi14230322-40:head by client.4568.0:214 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 35'15)", 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.646387+0000", 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.4596012859999998, 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10419487700000001, 2024-04-18T23:11:06.144 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646387+0000", 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646387+0000", 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.145 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646387+0000", 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7700000000000005e-07 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646405+0000", 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7722000000000001e-05 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646461+0000", 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6085999999999998e-05 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.746622+0000", 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10016080400000001 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.749532+0000", 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029097210000000001 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.750582+0000", 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001050067 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.5s5 40/31 ECSubWrite(tid=27, reqid=osd.15.0:26, at_version=0'0, trim_to=0'0, roll_forward_to=40'15))", 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:04.973085+0000", 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.1329025719999999, 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.11391169600000001, 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973085+0000", 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973085+0000", 2024-04-18T23:11:06.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973086+0000", 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2099999999999995e-07 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973103+0000", 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7042999999999999e-05 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973153+0000", 2024-04-18T23:11:06.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.003e-05 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973166+0000", 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2340999999999999e-05 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086086+0000", 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.112919777 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086100+0000", 2024-04-18T23:11:06.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4949999999999999e-05 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086924+0000", 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082378300000000002 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086997+0000", 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2850999999999998e-05 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 40/31 ECSubWrite(tid=81, reqid=osd.4.0:79, at_version=0'0, trim_to=0'0, roll_forward_to=40'33))", 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:04.973664+0000", 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.1323236880000001, 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102955542, 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973664+0000", 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973664+0000", 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973666+0000", 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973686+0000", 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9536000000000001e-05 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973723+0000", 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7302000000000003e-05 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973744+0000", 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0993e-05 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.073978+0000", 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10023391399999999 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.073996+0000", 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7612999999999999e-05 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.076565+0000", 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025693119999999998 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.076620+0000", 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4861999999999998e-05 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 40/31 ECSubWrite(tid=82, reqid=osd.4.0:82, at_version=40'35, trim_to=0'0, roll_forward_to=40'33))", 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:04.974058+0000", 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.1319301310000001, 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10443377099999999, 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974058+0000", 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974058+0000", 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974059+0000", 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7599999999999996e-07 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974104+0000", 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5636000000000001e-05 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974120+0000", 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5654999999999998e-05 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.074238+0000", 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10011843199999999 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.074461+0000", 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022277000000000001 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.074485+0000", 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3546000000000001e-05 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.078389+0000", 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039044829999999998 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.078492+0000", 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102373 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/31 ECSubWriteReply(tid=13, last_complete=40'8, committed=1, applied=1))", 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:04.976193+0000", 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.1297949300000001, 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0067378760000000003, 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.976193+0000", 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.976193+0000", 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.976194+0000", 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5700000000000001e-07 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.976200+0000", 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1369999999999999e-06 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.976214+0000", 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.437e-05 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.976224+0000", 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8989999999999996e-06 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.982087+0000", 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0058624139999999998 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.982931+0000", 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00084419900000000001 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/31 ECSubWriteReply(tid=14, last_complete=40'8, committed=1, applied=1))", 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:04.983983+0000", 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.1220045569999999, 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10025347499999999, 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.983983+0000", 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.983983+0000", 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.983985+0000", 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.477e-06 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.983993+0000", 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3799999999999994e-06 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.984017+0000", 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3359000000000001e-05 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.084102+0000", 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008505700000001 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.084207+0000", 2024-04-18T23:11:06.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010555 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.084237+0000", 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9652e-05 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.8s1 40/31 ECSubWrite(tid=17, reqid=osd.5.0:17, at_version=40'10, trim_to=0'0, roll_forward_to=36'8))", 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:05.082662+0000", 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.0233256019999999, 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101720165, 2024-04-18T23:11:06.166 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.082662+0000", 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.082662+0000", 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.082664+0000", 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4619999999999999e-06 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.082703+0000", 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9291e-05 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.082730+0000", 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7256000000000001e-05 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.182825+0000", 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10009417499999999 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.182996+0000", 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017106200000000001 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.183010+0000", 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.437e-05 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184289+0000", 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012785450000000001 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184383+0000", 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4004000000000003e-05 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 40/31 ECSubWriteReply(tid=14, last_complete=40'8, committed=1, applied=1))", 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:05.084260+0000", 2024-04-18T23:11:06.170 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.0217282999999999, 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100366117, 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.084260+0000", 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.084260+0000", 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.084261+0000", 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.778e-06 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.084270+0000", 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6910000000000003e-06 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.084291+0000", 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1107999999999998e-05 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184378+0000", 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008668900000001 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184567+0000", 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018916899999999999 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184626+0000", 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8681999999999998e-05 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.173 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpReadReply(3.6s0 40/31 ECSubReadReply(tid=15, attrs_read=0))", 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:05.175718+0000", 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.93026963900000004, 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0076345079999999999, 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.175718+0000", 2024-04-18T23:11:06.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.175718+0000", 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.175719+0000", 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9299999999999997e-07 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.175746+0000", 2024-04-18T23:11:06.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7279000000000001e-05 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.175766+0000", 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9510000000000001e-05 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.175779+0000", 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2656e-05 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.183276+0000", 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0074972179999999999 2024-04-18T23:11:06.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.183353+0000", 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7151999999999996e-05 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 40/31 ECSubWrite(tid=84, reqid=osd.4.0:83, at_version=0'0, trim_to=0'0, roll_forward_to=40'37))", 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:05.176176+0000", 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.92981238799999999, 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102118266, 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.177 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.176176+0000", 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.176176+0000", 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.176178+0000", 2024-04-18T23:11:06.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.99e-06 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.176203+0000", 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5276e-05 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.176235+0000", 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2199999999999997e-05 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.176254+0000", 2024-04-18T23:11:06.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9318999999999999e-05 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.176407+0000", 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015257899999999999 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.176433+0000", 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5539999999999999e-05 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.278265+0000", 2024-04-18T23:11:06.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101832295 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.278294+0000", 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9067000000000001e-05 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpReadReply(3.6s0 40/31 ECSubReadReply(tid=15, attrs_read=0))", 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:05.184437+0000", 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.92155116199999998, 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.091770229999999994, 2024-04-18T23:11:06.181 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184437+0000", 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184437+0000", 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184438+0000", 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1039999999999999e-06 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184460+0000", 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2106000000000001e-05 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184484+0000", 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3672999999999998e-05 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.184499+0000", 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5569e-05 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.275885+0000", 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.091385776000000002 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.276207+0000", 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000322002 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.8s1 40/31 ECSubWrite(tid=19, reqid=osd.5.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:05.187735+0000", 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.91825327700000003, 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201244059, 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187735+0000", 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187735+0000", 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187741+0000", 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4200000000000004e-06 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187762+0000", 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1095999999999999e-05 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187791+0000", 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8875999999999999e-05 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.287866+0000", 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100074872 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388009+0000", 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100142971 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388016+0000", 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5919999999999997e-06 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388929+0000", 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00091366799999999997 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388979+0000", 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9564000000000002e-05 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.188 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.189 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:06.189 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-18T23:11:06.189 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-18T23:11:06.189 INFO:tasks.thrashosds.thrasher:trim_stale_osdmaps 2024-04-18T23:11:06.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 'trim stale osdmaps' 2024-04-18T23:11:06.191 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:06.191+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:06.191 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:11:06.291 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:06.292+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:06.364 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:06.364 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:11:06.364 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:06.380 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:11:06.392 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:06.391+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:06.403 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:06.405 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:06.406 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:06.406 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 'trim stale osdmaps' 2024-04-18T23:11:06.493 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:06.488+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:06.494 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:06.494 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:06.495 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:06.509 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:11:06.578 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-04-18T23:11:06.578 INFO:tasks.rados.rados.0.smithi142.stdout:102: done (0 left) 2024-04-18T23:11:06.579 INFO:tasks.rados.rados.0.smithi142.stdout:103: read oid 29 snap 4 2024-04-18T23:11:06.579 INFO:tasks.rados.rados.0.smithi142.stdout:103: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-18T23:11:06.579 INFO:tasks.rados.rados.0.smithi142.stdout:104: read oid 46 snap -1 2024-04-18T23:11:06.579 INFO:tasks.rados.rados.0.smithi142.stdout:104: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-18T23:11:06.579 INFO:tasks.rados.rados.0.smithi142.stdout:105: copy_from oid 36 from oid 2 current snap is 5 2024-04-18T23:11:06.579 INFO:tasks.rados.rados.0.smithi142.stdout:106: rmattr oid 49 current snap is 5 2024-04-18T23:11:06.579 INFO:tasks.rados.rados.0.smithi142.stdout:106: done (3 left) 2024-04-18T23:11:06.579 INFO:tasks.rados.rados.0.smithi142.stdout:107: snap_remove snap 3 2024-04-18T23:11:06.594 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:06.592+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:06.614 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:06.614 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:06.614 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:06.614 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:06.614 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:06.614 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:06.617 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:06.617 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:06.617 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 'trim stale osdmaps' 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:1 3.1s0 3:8af79e0a:::smithi14230322-1:head [append 0~32768,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.514184+0000", 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.159456104, 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30151697599999999, 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.675 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 1 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514184+0000", 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514184+0000", 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514188+0000", 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3900000000000002e-06 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514308+0000", 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119943 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514343+0000", 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5029e-05 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514373+0000", 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0300000000000001e-05 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514686+0000", 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000312916 2024-04-18T23:11:06.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514963+0000", 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00027705299999999998 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.519925+0000", 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0049618320000000002 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625046+0000", 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10512126400000001 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.815573+0000", 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.190526636 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.815701+0000", 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000128613 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:2 3.1s0 3:8af79e0a:::smithi14230322-1:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.514435+0000", 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.15920582, 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30788222399999998, 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 2 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514435+0000", 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514435+0000", 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514436+0000", 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.716e-06 2024-04-18T23:11:06.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514448+0000", 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1440000000000001e-05 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514491+0000", 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3115999999999999e-05 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.514512+0000", 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0882000000000001e-05 2024-04-18T23:11:06.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.522737+0000", 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0082250280000000005 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.522987+0000", 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025047999999999997 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.524423+0000", 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014358680000000001 2024-04-18T23:11:06.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725217+0000", 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20079397700000001 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.822191+0000", 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.096973769000000001 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.822317+0000", 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012594799999999999 2024-04-18T23:11:06.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:16 3.7s0 3:e60e3c0c:::smithi14230322-6:head [append 0~98304,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.515790+0000", 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.157850337999999, 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.21435690900000001, 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.686 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 16 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.515790+0000", 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.515790+0000", 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.515793+0000", 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4839999999999998e-06 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516021+0000", 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022792099999999999 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516046+0000", 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4922999999999998e-05 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516067+0000", 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1569e-05 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516172+0000", 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010441 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516342+0000", 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000170125 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517682+0000", 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001340281 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.525122+0000", 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0074399460000000002 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729920+0000", 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20479840199999999 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.730147+0000", 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022684800000000001 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:17 3.7s0 3:e60e3c0c:::smithi14230322-6:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.516119+0000", 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.157521459, 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.214640478, 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.691 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 17 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516119+0000", 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516119+0000", 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516121+0000", 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.587e-06 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516129+0000", 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3999999999999992e-06 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516150+0000", 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0695000000000001e-05 2024-04-18T23:11:06.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516165+0000", 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5546999999999999e-05 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523484+0000", 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0073189969999999998 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523604+0000", 2024-04-18T23:11:06.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119312 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.524325+0000", 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00072162300000000001 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.629424+0000", 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10509901000000001 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.730518+0000", 2024-04-18T23:11:06.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10109409599999999 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.730760+0000", 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024121099999999999 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:19 3.bs0 3:d498df52:::smithi14230322-7:head [append 0~114688,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.516777+0000", 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.156863305, 2024-04-18T23:11:06.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20870124600000001, 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 19 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516777+0000", 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516777+0000", 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.516779+0000", 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.663e-06 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517128+0000", 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034904499999999998 2024-04-18T23:11:06.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517157+0000", 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8711999999999999e-05 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517177+0000", 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0356e-05 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517265+0000", 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7979000000000006e-05 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517422+0000", 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000156904 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523148+0000", 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.005725646 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628334+0000", 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.105186686 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725244+0000", 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.096909334 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725478+0000", 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000234921 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.701 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:20 3.bs0 3:d498df52:::smithi14230322-7:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.517227+0000", 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.156413880000001, 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30820565100000002, 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 20 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517227+0000", 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517227+0000", 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517228+0000", 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.64e-06 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517238+0000", 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6129999999999993e-06 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517257+0000", 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8763e-05 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517272+0000", 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5003e-05 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.524254+0000", 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069820419999999999 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.524480+0000", 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000226271 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.525717+0000", 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012373219999999999 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628337+0000", 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102620139 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.825312+0000", 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19697435099999999 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.825432+0000", 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120507 2024-04-18T23:11:06.706 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:21 3.bs0 3:d498df52:::smithi14230322-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.517296+0000", 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.156344528, 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.31922700999999998, 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.707 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517296+0000", 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517296+0000", 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517297+0000", 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3859999999999999e-06 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517303+0000", 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.237e-06 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517327+0000", 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4263000000000001e-05 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517342+0000", 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4806e-05 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628177+0000", 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.110835323 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628320+0000", 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014254200000000001 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.825855+0000", 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.197534974 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.826053+0000", 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000198927 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.836523+0000", 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010469552 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:181 3.bs0 3:dd01ba5f:::smithi14230322-23:head [append 0~524288] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.552123+0000", 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.1215171350000004, 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.31423346800000002, 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 181 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552123+0000", 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552123+0000", 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552126+0000", 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6690000000000002e-06 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552796+0000", 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00066951100000000002 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552835+0000", 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9122000000000003e-05 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552847+0000", 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2509e-05 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552926+0000", 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8658000000000005e-05 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552932+0000", 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8780000000000003e-06 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.657708+0000", 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10477632200000001 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.657762+0000", 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3603000000000002e-05 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658408+0000", 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064599500000000001 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757880+0000", 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099471941999999994 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.866304+0000", 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.108424836 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.866357+0000", 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2423000000000003e-05 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:182 3.bs0 3:dd01ba5f:::smithi14230322-23:head [append 0~376832] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.717 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.557068+0000", 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.1165727490000004, 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30940973100000002, 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 182 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557068+0000", 2024-04-18T23:11:06.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557068+0000", 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557069+0000", 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4640000000000001e-06 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557412+0000", 2024-04-18T23:11:06.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00034307700000000001 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557427+0000", 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4847e-05 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557436+0000", 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8210000000000008e-06 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557496+0000", 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0307999999999999e-05 2024-04-18T23:11:06.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557502+0000", 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1160000000000004e-06 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.659032+0000", 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101529191 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.659080+0000", 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7929999999999997e-05 2024-04-18T23:11:06.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.659485+0000", 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00040593200000000002 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757881+0000", 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.098395094000000002 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.866424+0000", 2024-04-18T23:11:06.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.108543906 2024-04-18T23:11:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.866478+0000", 2024-04-18T23:11:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3044999999999998e-05 2024-04-18T23:11:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.725 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:183 3.bs0 3:dd01ba5f:::smithi14230322-23:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.557454+0000", 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.1161861890000004, 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40969022799999999, 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 183 2024-04-18T23:11:06.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557454+0000", 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557454+0000", 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557455+0000", 2024-04-18T23:11:06.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.89e-07 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557459+0000", 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8070000000000001e-06 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557466+0000", 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9e-06 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557470+0000", 2024-04-18T23:11:06.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4440000000000001e-06 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557529+0000", 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8903999999999997e-05 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557532+0000", 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6579999999999998e-06 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.659801+0000", 2024-04-18T23:11:06.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102268964 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.659849+0000", 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8466e-05 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.660148+0000", 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029888700000000001 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757882+0000", 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097734330999999994 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967050+0000", 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20916807400000001 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967145+0000", 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4303999999999997e-05 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:184 3.bs0 3:dd01ba5f:::smithi14230322-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.1161496270000004, 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61588253999999998, 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 184 2024-04-18T23:11:06.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8700000000000001e-07 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557493+0000", 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9149999999999999e-06 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557499+0000", 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2650000000000002e-06 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557504+0000", 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5689999999999998e-06 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654492+0000", 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.096987633000000004 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654498+0000", 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5860000000000001e-06 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757767+0000", 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103269131 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757839+0000", 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1878999999999994e-05 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967626+0000", 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.209786891 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967877+0000", 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025035699999999999 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173373+0000", 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.205496927 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:188 3.7s0 3:e5b49aa4:::smithi14230322-32:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558569+0000", 2024-04-18T23:11:06.737 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.1150712619999998, 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50916866999999999, 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 188 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558569+0000", 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558569+0000", 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558570+0000", 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2499999999999995e-07 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558621+0000", 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1082999999999999e-05 2024-04-18T23:11:06.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558629+0000", 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9529999999999999e-06 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558634+0000", 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7360000000000001e-06 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558665+0000", 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1099000000000002e-05 2024-04-18T23:11:06.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558669+0000", 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3610000000000003e-06 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654665+0000", 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.095996355000000005 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654801+0000", 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013551599999999999 2024-04-18T23:11:06.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.655971+0000", 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011700059999999999 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.657081+0000", 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001109737 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067667+0000", 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41058618899999999 2024-04-18T23:11:06.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067738+0000", 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-05 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:189 3.bs0 3:d498df52:::smithi14230322-7:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.1149573449999997, 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.91155819000000005, 2024-04-18T23:11:06.743 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 189 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558684+0000", 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1799999999999995e-07 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558687+0000", 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1920000000000001e-06 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558694+0000", 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.452e-06 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658767+0000", 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100072129 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757920+0000", 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099153626999999994 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757967+0000", 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6962999999999997e-05 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171274+0000", 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41330696700000003 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173905+0000", 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002630828 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470100+0000", 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29619521399999998 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470241+0000", 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014119999999999999 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:190 3.bs0 3:d498df52:::smithi14230322-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.658782+0000", 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.0148584950000004, 2024-04-18T23:11:06.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.51341810700000001, 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 190 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658782+0000", 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658782+0000", 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658783+0000", 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1209999999999999e-06 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658789+0000", 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658798+0000", 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6099999999999995e-06 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658805+0000", 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4489999999999996e-06 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760577+0000", 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101772846 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760624+0000", 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6789000000000002e-05 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.172000+0000", 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41137571899999997 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.172095+0000", 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4592000000000005e-05 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.172200+0000", 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105546 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:196 3.7s0 3:e60e3c0c:::smithi14230322-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.658830+0000", 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.0148106539999997, 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40905263600000002, 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.753 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 196 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658830+0000", 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658830+0000", 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658831+0000", 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.37e-07 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658834+0000", 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4589999999999998e-06 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658841+0000", 2024-04-18T23:11:06.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2629999999999997e-06 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758909+0000", 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100068197 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758967+0000", 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759068+0000", 2024-04-18T23:11:06.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101267 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759746+0000", 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00067788199999999999 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863226+0000", 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103479856 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067834+0000", 2024-04-18T23:11:06.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20460830399999999 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067883+0000", 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8195999999999997e-05 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:197 3.7s0 3:e60e3c0c:::smithi14230322-6:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.758927+0000", 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.9147140299999998, 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40981388400000002, 2024-04-18T23:11:06.758 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 197 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758927+0000", 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758927+0000", 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758928+0000", 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0470000000000001e-06 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758932+0000", 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758939+0000", 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9280000000000004e-06 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758944+0000", 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4700000000000004e-06 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760155+0000", 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001211073 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760309+0000", 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015378399999999999 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760963+0000", 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00065410399999999997 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863295+0000", 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102332388 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168687+0000", 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30539211500000002 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168740+0000", 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3152999999999997e-05 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:198 3.7s0 3:e60e3c0c:::smithi14230322-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:06.763 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.9146829040000002, 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41079902800000001, 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 198 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9799999999999999e-07 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758960+0000", 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.623e-06 2024-04-18T23:11:06.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758964+0000", 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4819999999999996e-06 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758967+0000", 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4570000000000001e-06 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761240+0000", 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002272081 2024-04-18T23:11:06.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761347+0000", 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010780099999999999 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761749+0000", 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00040184299999999998 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863298+0000", 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101549268 2024-04-18T23:11:06.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169722+0000", 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.306423995 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169757+0000", 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4180000000000001e-05 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.768 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:212 3.7s0 3:e405d834:::smithi14230322-14:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.9146675650000002, 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41097265900000002, 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 212 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9099999999999999e-07 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759003+0000", 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9641999999999998e-05 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759008+0000", 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.271e-06 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759012+0000", 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1359999999999999e-06 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.862010+0000", 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102997908 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.862080+0000", 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9305000000000001e-05 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.862724+0000", 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064440199999999997 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863536+0000", 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081221099999999997 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169861+0000", 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30632486799999997 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169946+0000", 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4524999999999995e-05 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.7.22:1 3.1s0 3:8af79e0a:::smithi14230322-1:head [assert-version v2,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.761350+0000", 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.9122908289999998, 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30728936000000001, 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761350+0000", 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761350+0000", 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761351+0000", 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1480000000000001e-06 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761356+0000", 2024-04-18T23:11:06.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.592e-06 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761365+0000", 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5059999999999995e-06 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761370+0000", 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2800000000000003e-06 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.861475+0000", 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100104887 2024-04-18T23:11:06.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.861542+0000", 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7059999999999998e-05 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068639+0000", 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20709688700000001 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.777 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.bs0 36/31 ECSubWriteReply(tid=57, last_complete=36'15, committed=1, applied=1))", 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.766292+0000", 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.9073488850000002, 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.200013527, 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766292+0000", 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766292+0000", 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766292+0000", 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.48e-07 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766294+0000", 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8700000000000001e-06 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766301+0000", 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9e-06 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.866364+0000", 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10006341000000001 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.966293+0000", 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099929258000000007 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:06.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.966305+0000", 2024-04-18T23:11:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1841000000000001e-05 2024-04-18T23:11:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.781 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.781 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:06.782 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:11:06.783 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:06.783+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:06.819 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:06.819 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:06.820 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:06.822 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:06.822 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:06.823 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 'trim stale osdmaps' 2024-04-18T23:11:06.883 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:06.880+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.4.21:2 3.4s0 3:29b33eb5:::smithi14230322-2:head [assert-version v15,copy-get max 8388608 in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.887375+0000", 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.043150521999999997, 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.043187288999999997, 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:06.930 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887375+0000", 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887375+0000", 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887376+0000", 2024-04-18T23:11:06.931 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8279999999999999e-06 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887382+0000", 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9270000000000002e-06 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887395+0000", 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2757e-05 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887404+0000", 2024-04-18T23:11:06.932 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5420000000000005e-06 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887540+0000", 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013645399999999999 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887655+0000", 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000114558 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.933 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpReadReply(3.4s0 42/31 ECSubReadReply(tid=29, attrs_read=0))", 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.900046+0000", 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.030479301, 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030563896, 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "no flag points reached", 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.900046+0000", 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.934 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.900046+0000", 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.900047+0000", 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2699999999999998e-07 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.900484+0000", 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00043722900000000002 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:06.935 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.900498+0000", 2024-04-18T23:11:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4283e-05 2024-04-18T23:11:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-18T23:11:06.936 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 2 2024-04-18T23:11:06.936 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:06.943 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:11:06.983 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:06.984+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:07.023 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:07.024 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:07.024 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 'trim stale osdmaps' 2024-04-18T23:11:07.063 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:07.064 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:07.064 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:07.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:11:07.084 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:07.083+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:07.181 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:07.181 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:07.181 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:07.181 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:07.181 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:07.182 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:07.184 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:07.184 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:07.184 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 'trim stale osdmaps' 2024-04-18T23:11:07.186 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:07.183+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:07.242 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.9s5 34/31 ECSubWrite(tid=1, reqid=client.4568.0:8, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.242 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.528882+0000", 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.712431801999999, 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104398464, 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.528882+0000", 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.528882+0000", 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.528883+0000", 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0669999999999999e-06 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.528908+0000", 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5048000000000001e-05 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.528928+0000", 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9573e-05 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.528939+0000", 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1396000000000001e-05 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.629233+0000", 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10029334099999999 2024-04-18T23:11:07.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.629261+0000", 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8291999999999999e-05 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.633227+0000", 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039658209999999996 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.633281+0000", 2024-04-18T23:11:07.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3925999999999998e-05 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds5 34/31 ECSubWrite(tid=0, reqid=client.4568.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.532278+0000", 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.709036308, 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20155427200000001, 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532278+0000", 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532278+0000", 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532278+0000", 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7500000000000001e-07 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532500+0000", 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022245699999999999 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532515+0000", 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4592e-05 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532527+0000", 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2218999999999999e-05 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.632678+0000", 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100150425 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.632686+0000", 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7609999999999996e-06 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.733761+0000", 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10107475 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.733832+0000", 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0692999999999996e-05 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es4 34/31 ECSubWrite(tid=0, reqid=client.4568.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.533099+0000", 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.708214883, 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103224209, 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.251 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.533099+0000", 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.533099+0000", 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.533099+0000", 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.86e-07 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.533398+0000", 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029876600000000001 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.533413+0000", 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4525999999999999e-05 2024-04-18T23:11:07.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.533420+0000", 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.374e-06 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.633546+0000", 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100126059 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.633555+0000", 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5049999999999996e-06 2024-04-18T23:11:07.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.636272+0000", 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002716619 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.636323+0000", 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1174000000000002e-05 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.255 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es4 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.535541+0000", 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.705773342000001, 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10080753000000001, 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.535541+0000", 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.535541+0000", 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.535542+0000", 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2139999999999999e-06 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.535559+0000", 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7269e-05 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.535574+0000", 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4632e-05 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.535593+0000", 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9168e-05 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.634064+0000", 2024-04-18T23:11:07.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.098470905999999997 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.634074+0000", 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8649999999999996e-06 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.636328+0000", 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002254074 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.636348+0000", 2024-04-18T23:11:07.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0401999999999999e-05 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.624985+0000", 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.616329114999999, 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100989736, 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624985+0000", 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624985+0000", 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624986+0000", 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.178e-06 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624997+0000", 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1302e-05 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625028+0000", 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0439000000000001e-05 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725146+0000", 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100117996 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725940+0000", 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079469999999999996 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725975+0000", 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4121000000000001e-05 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:07.263 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.628124+0000", 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.61318954, 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20045701599999999, 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628124+0000", 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628124+0000", 2024-04-18T23:11:07.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628125+0000", 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2200000000000002e-07 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628131+0000", 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2530000000000004e-06 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628145+0000", 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4430000000000001e-05 2024-04-18T23:11:07.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728230+0000", 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100084688 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.828536+0000", 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10030650100000001 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.828581+0000", 2024-04-18T23:11:07.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5222000000000002e-05 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=3, last_complete=34'4, committed=1, applied=1))", 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.726428+0000", 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.514886062, 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10202667999999999, 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726428+0000", 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726428+0000", 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726429+0000", 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2600000000000001e-07 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726434+0000", 2024-04-18T23:11:07.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.164e-06 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726446+0000", 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2437e-05 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726454+0000", 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7519999999999996e-06 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.828419+0000", 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10196517400000001 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.828455+0000", 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5327000000000002e-05 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=2, last_complete=34'3, committed=1, applied=1))", 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.728315+0000", 2024-04-18T23:11:07.270 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.512999018, 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20072921299999999, 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728315+0000", 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728315+0000", 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728316+0000", 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000001e-07 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728321+0000", 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9799999999999998e-06 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728339+0000", 2024-04-18T23:11:07.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8584e-05 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728352+0000", 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3124e-05 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.929018+0000", 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20066600000000001 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.929044+0000", 2024-04-18T23:11:07.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5908999999999999e-05 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.729293+0000", 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.512021398, 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19966805300000001, 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729293+0000", 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729293+0000", 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729293+0000", 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8400000000000003e-07 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729302+0000", 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5620000000000004e-06 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729315+0000", 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3263e-05 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729327+0000", 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1592e-05 2024-04-18T23:11:07.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.928932+0000", 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.199605167 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.928961+0000", 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8585000000000002e-05 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.277 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/31 ECSubWriteReply(tid=2, last_complete=34'3, committed=1, applied=1))", 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.729347+0000", 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.511967158999999, 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29994712699999998, 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729347+0000", 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729347+0000", 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729347+0000", 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5000000000000003e-07 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729351+0000", 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8480000000000003e-06 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729364+0000", 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3171e-05 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729374+0000", 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1779999999999992e-06 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.029281+0000", 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29990735099999999 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.029294+0000", 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3029e-05 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.732253+0000", 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.509061312, 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196610745, 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.281 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732253+0000", 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732253+0000", 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732254+0000", 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2920000000000001e-06 2024-04-18T23:11:07.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732262+0000", 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6320000000000008e-06 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732285+0000", 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3434e-05 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732300+0000", 2024-04-18T23:11:07.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5390999999999999e-05 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.928829+0000", 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19652907 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.928863+0000", 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3926e-05 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.732325+0000", 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.508989049, 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29692952, 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732325+0000", 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732325+0000", 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732325+0000", 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7199999999999999e-07 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732330+0000", 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3470000000000001e-06 2024-04-18T23:11:07.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732343+0000", 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2816000000000001e-05 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732352+0000", 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1600000000000004e-06 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.029234+0000", 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29688247800000001 2024-04-18T23:11:07.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.029254+0000", 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0146999999999999e-05 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.9s5 34/31 ECSubWrite(tid=4, reqid=client.4568.0:8, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.734963+0000", 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.506350714, 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10253271999999999, 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.288 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.734963+0000", 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.734963+0000", 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.734964+0000", 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0100000000000003e-07 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.734976+0000", 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1922999999999999e-05 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.734993+0000", 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6806000000000001e-05 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.835157+0000", 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100164588 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.835987+0000", 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082944300000000002 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.836012+0000", 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5179999999999999e-05 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.837407+0000", 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013951949999999999 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.837496+0000", 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8684000000000007e-05 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:194 3.4s0 3:2a56ff57:::smithi14230322-24:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:07.292 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.6825346049999998, 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41191497500000002, 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 194 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558781+0000", 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8059999999999999e-06 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558789+0000", 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1340000000000004e-06 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558806+0000", 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6300999999999999e-05 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558814+0000", 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0099999999999995e-06 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558882+0000", 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8281999999999994e-05 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558889+0000", 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2359999999999998e-06 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661030+0000", 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102141155 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:07.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661144+0000", 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113853 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766467+0000", 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.105323126 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.768352+0000", 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018843359999999999 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.970633+0000", 2024-04-18T23:11:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.202281391 2024-04-18T23:11:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.970694+0000", 2024-04-18T23:11:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1345000000000003e-05 2024-04-18T23:11:07.303 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:195 3.4s0 3:2a56ff57:::smithi14230322-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558830+0000", 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.6824840779999999, 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20893925299999999, 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:07.304 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 195 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558830+0000", 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558830+0000", 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558830+0000", 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6699999999999999e-07 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558833+0000", 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9050000000000001e-06 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558838+0000", 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5970000000000002e-06 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558848+0000", 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0351e-05 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558961+0000", 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011244299999999999 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558970+0000", 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7229999999999999e-06 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661751+0000", 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102780887 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661782+0000", 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1164999999999997e-05 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.767573+0000", 2024-04-18T23:11:07.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10579037500000001 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.767662+0000", 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9672000000000005e-05 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.767769+0000", 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010676800000000001 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.309 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.5.21:1 3.4s0 3:29b33eb5:::smithi14230322-2:head [assert-version v6,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.643438+0000", 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.5978758930000003, 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.119307518, 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.643438+0000", 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.643438+0000", 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.643439+0000", 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3689999999999999e-06 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.643452+0000", 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2434e-05 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.643462+0000", 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8570000000000007e-06 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.643474+0000", 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2339e-05 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.643539+0000", 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5129000000000006e-05 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.643544+0000", 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0649999999999998e-06 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.660499+0000", 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016954642999999998 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.660614+0000", 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115262 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.313 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.762746+0000", 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10213142 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 36/31 ECSubWriteReply(tid=18, last_complete=36'11, committed=1, applied=1))", 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.972289+0000", 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.2690253309999999, 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30040967099999999, 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.314 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972289+0000", 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972289+0000", 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972289+0000", 2024-04-18T23:11:07.315 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8500000000000002e-07 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972292+0000", 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6970000000000002e-06 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972298+0000", 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4110000000000001e-06 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.072395+0000", 2024-04-18T23:11:07.316 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100096515 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.272633+0000", 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20023834400000001 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.272698+0000", 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5219e-05 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.317 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s4 36/31 ECSubWrite(tid=65, reqid=client.4568.0:212, at_version=0'0, trim_to=0'0, roll_forward_to=36'25))", 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.170123+0000", 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.0711911839999999, 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10140460599999999, 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170123+0000", 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.318 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170123+0000", 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170124+0000", 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.299e-06 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170138+0000", 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4005000000000001e-05 2024-04-18T23:11:07.319 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.170159+0000", 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1243e-05 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.270255+0000", 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100095541 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.270365+0000", 2024-04-18T23:11:07.320 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010975400000000001 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.270375+0000", 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0783e-05 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.271476+0000", 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001100354 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.321 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.271527+0000", 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1626999999999999e-05 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "pg_update_log_missing(3.9s5 epoch 37/31 rep_tid 41 entries 37'17 (0'0) error 3:9e84ac80:::smithi14230322-40:head by client.4568.0:214 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 35'15)", 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.646494+0000", 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.5948201920000002, 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101360095, 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.322 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646494+0000", 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646494+0000", 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646494+0000", 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7000000000000005e-07 2024-04-18T23:11:07.323 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646506+0000", 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1922999999999999e-05 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646549+0000", 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2919e-05 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646581+0000", 2024-04-18T23:11:07.324 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1433000000000002e-05 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.746858+0000", 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100277745 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.747854+0000", 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00099550500000000004 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.325 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es4 42/31 ECSubWrite(tid=24, reqid=osd.2.0:24, at_version=42'14, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.593861+0000", 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.64745301499999997, 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20227319099999999, 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593861+0000", 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.326 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593861+0000", 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593861+0000", 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6899999999999999e-07 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593893+0000", 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2258999999999997e-05 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.327 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593903+0000", 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2150000000000007e-06 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.693989+0000", 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008618499999999 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.795360+0000", 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10137168000000001 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.328 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.795374+0000", 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3842000000000001e-05 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.796103+0000", 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00072852400000000003 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.796134+0000", 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1316999999999998e-05 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.329 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.330 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.330 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:07.331 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:11:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:07.333 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:07.332+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:07.334 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:07.334 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:07.335 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 'trim stale osdmaps' 2024-04-18T23:11:07.431 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:07.426+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:07.445 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:07.445 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:11:07.445 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:07.460 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:11:07.465 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:07.523 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:07.524 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:07.527 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:07.527 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:07.527 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 'trim stale osdmaps' 2024-04-18T23:11:07.532 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:07.532+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:07.586 INFO:tasks.rados.rados.0.smithi142.stdout:107: done (3 left) 2024-04-18T23:11:07.586 INFO:tasks.rados.rados.0.smithi142.stdout:108: rmattr oid 30 current snap is 5 2024-04-18T23:11:07.586 INFO:tasks.rados.rados.0.smithi142.stdout:108: done (3 left) 2024-04-18T23:11:07.586 INFO:tasks.rados.rados.0.smithi142.stdout:109: read oid 42 snap 2 2024-04-18T23:11:07.587 INFO:tasks.rados.rados.0.smithi142.stdout:109: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-18T23:11:07.587 INFO:tasks.rados.rados.0.smithi142.stdout:110: rmattr oid 14 current snap is 5 2024-04-18T23:11:07.587 INFO:tasks.rados.rados.0.smithi142.stdout:104: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-18T23:11:07.623 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:07.623 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:07.623 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:07.623 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:07.624 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:07.633 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:07.630+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:07.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:11:07.643 INFO:tasks.rados.rados.0.smithi142.stdout:105: finishing copy_from racing read to smithi14230322-36 2024-04-18T23:11:07.643 INFO:tasks.rados.rados.0.smithi142.stdout:104: done (4 left) 2024-04-18T23:11:07.643 INFO:tasks.rados.rados.0.smithi142.stdout:111: read oid 9 snap 2 2024-04-18T23:11:07.643 INFO:tasks.rados.rados.0.smithi142.stdout:111: expect (ObjNum 8 snap 0 seq_num 8) 2024-04-18T23:11:07.643 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 25 current snap is 5 2024-04-18T23:11:07.643 INFO:tasks.rados.rados.0.smithi142.stdout:112: seq_num 59 ranges {1261568=622592,1884160=376832} 2024-04-18T23:11:07.656 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:07.656 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:07.674 INFO:tasks.rados.rados.0.smithi142.stdout:112: writing smithi14230322-25 from 1261568 to 1884160 tid 1 2024-04-18T23:11:07.681 INFO:tasks.rados.rados.0.smithi142.stdout:112: writing smithi14230322-25 from 1884160 to 2260992 tid 2 2024-04-18T23:11:07.681 INFO:tasks.rados.rados.0.smithi142.stdout:113: copy_from oid 32 from oid 6 current snap is 5 2024-04-18T23:11:07.681 INFO:tasks.rados.rados.0.smithi142.stdout:105: finishing copy_from to smithi14230322-36 2024-04-18T23:11:07.681 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 36 v 15 (ObjNum 56 snap 4 seq_num 56) dirty exists 2024-04-18T23:11:07.681 INFO:tasks.rados.rados.0.smithi142.stdout:103: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-18T23:11:07.708 INFO:tasks.rados.rados.0.smithi142.stdout:103: done (6 left) 2024-04-18T23:11:07.708 INFO:tasks.rados.rados.0.smithi142.stdout:105: done (5 left) 2024-04-18T23:11:07.709 INFO:tasks.rados.rados.0.smithi142.stdout:114: snap_remove snap 4 2024-04-18T23:11:07.730 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:07.730 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:07.731 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:07.733 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:07.734 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:07.734 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 'trim stale osdmaps' 2024-04-18T23:11:07.735 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:07.732+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:07.750 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:07.751 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/31 ECSubWrite(tid=2, reqid=client.4568.0:19, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.244534128, 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102238363, 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.769 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8599999999999998e-07 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523292+0000", 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2746e-05 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523307+0000", 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5223e-05 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623426+0000", 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10011853599999999 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623593+0000", 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016698900000000001 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623604+0000", 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1829e-05 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624978+0000", 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013738699999999999 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625487+0000", 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00050868400000000002 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/31 ECSubWrite(tid=5, reqid=client.4568.0:20, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.773 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.623497+0000", 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.144286105999999, 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020570050000000002, 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623497+0000", 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623497+0000", 2024-04-18T23:11:07.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623498+0000", 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.204e-06 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623529+0000", 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0932000000000001e-05 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623551+0000", 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2303e-05 2024-04-18T23:11:07.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623587+0000", 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5735999999999998e-05 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624027+0000", 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00044001100000000003 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624035+0000", 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3690000000000007e-06 2024-04-18T23:11:07.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625494+0000", 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014592489999999999 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625554+0000", 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9200999999999998e-05 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 34/31 ECSubWrite(tid=0, reqid=client.4568.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.777 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.623754+0000", 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.144028832, 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102510163, 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623754+0000", 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623754+0000", 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623755+0000", 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5000000000000002e-07 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623927+0000", 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000172802 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623957+0000", 2024-04-18T23:11:07.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9016000000000001e-05 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623977+0000", 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0886e-05 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.724391+0000", 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100413961 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.724405+0000", 2024-04-18T23:11:07.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4079e-05 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726222+0000", 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001816151 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726264+0000", 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2518000000000002e-05 2024-04-18T23:11:07.781 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.143754719, 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10419479800000001, 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624029+0000", 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4300000000000001e-07 2024-04-18T23:11:07.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624060+0000", 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1025999999999997e-05 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624083+0000", 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3323e-05 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624104+0000", 2024-04-18T23:11:07.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1089000000000001e-05 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725186+0000", 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101081146 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725195+0000", 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7829999999999993e-06 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728164+0000", 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029689249999999999 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728223+0000", 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8563000000000002e-05 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.6s4 34/31 ECSubWrite(tid=0, reqid=client.4568.0:34, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.786 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.731401+0000", 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.036381626000001, 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10113056600000001, 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731401+0000", 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731401+0000", 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731402+0000", 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2599999999999998e-07 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731498+0000", 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6448999999999998e-05 2024-04-18T23:11:07.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731528+0000", 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9924999999999999e-05 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731544+0000", 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6500000000000001e-05 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.831700+0000", 2024-04-18T23:11:07.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100155789 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.831710+0000", 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0006e-05 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.832492+0000", 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00078151200000000005 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.832532+0000", 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9959000000000001e-05 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.6s4 34/31 ECSubWrite(tid=1, reqid=client.4568.0:35, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.834772+0000", 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.933011164, 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031159590000000002, 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.834772+0000", 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.834772+0000", 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.834772+0000", 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.54e-07 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.834797+0000", 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4369e-05 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.834818+0000", 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0948999999999999e-05 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.834828+0000", 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0233e-05 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.834914+0000", 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6464000000000004e-05 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.834923+0000", 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8810000000000002e-06 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.837810+0000", 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028862499999999999 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.837888+0000", 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7959000000000004e-05 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 34/31 ECSubWrite(tid=2, reqid=client.4568.0:38, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.930202+0000", 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.837580658, 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0015372109999999999, 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.795 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.930202+0000", 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.930202+0000", 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.930204+0000", 2024-04-18T23:11:07.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.95e-06 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.930253+0000", 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8887999999999997e-05 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.930287+0000", 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3908999999999997e-05 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.930304+0000", 2024-04-18T23:11:07.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7027000000000001e-05 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.930470+0000", 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016588599999999999 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.930497+0000", 2024-04-18T23:11:07.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7288999999999999e-05 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.931648+0000", 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001150444 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.931739+0000", 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1817999999999994e-05 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 34/31 ECSubWrite(tid=4, reqid=client.4568.0:49, at_version=34'3, trim_to=0'0, roll_forward_to=34'2))", 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.933551+0000", 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.834231802, 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020938039999999999, 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.933551+0000", 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.933551+0000", 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.933553+0000", 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7630000000000001e-06 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.933808+0000", 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025506299999999998 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.933835+0000", 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7506999999999999e-05 2024-04-18T23:11:07.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.933851+0000", 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5418999999999999e-05 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.934025+0000", 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017385299999999999 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.934042+0000", 2024-04-18T23:11:07.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7521999999999999e-05 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.935586+0000", 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001543979 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.935645+0000", 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8697999999999999e-05 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 34/31 ECSubWrite(tid=0, reqid=client.4568.0:58, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.941896+0000", 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.825886408000001, 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020127600000000002, 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.941896+0000", 2024-04-18T23:11:07.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.941896+0000", 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.941897+0000", 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3500000000000002e-07 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942131+0000", 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000234434 2024-04-18T23:11:07.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942152+0000", 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1056e-05 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942168+0000", 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6271000000000001e-05 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942286+0000", 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117791 2024-04-18T23:11:07.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942301+0000", 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4439e-05 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.943868+0000", 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001567195 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.943909+0000", 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1239000000000003e-05 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.807 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 34/31 ECSubWrite(tid=1, reqid=client.4568.0:59, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.942694+0000", 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.825088601000001, 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001651549, 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942694+0000", 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942694+0000", 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942696+0000", 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2029999999999999e-06 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942735+0000", 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9032999999999997e-05 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942770+0000", 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4768999999999999e-05 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.942795+0000", 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4465999999999999e-05 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.943052+0000", 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025702599999999999 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.943070+0000", 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8603000000000001e-05 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944318+0000", 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001248173 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944346+0000", 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7276000000000001e-05 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 34/31 ECSubWrite(tid=2, reqid=client.4568.0:61, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.944827+0000", 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.822955783999999, 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10111951700000001, 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944827+0000", 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944827+0000", 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944828+0000", 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.257e-06 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944993+0000", 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016476899999999999 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.945018+0000", 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222999999999999e-05 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.945033+0000", 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4859e-05 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045160+0000", 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100126494 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045166+0000", 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3539999999999996e-06 2024-04-18T23:11:07.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045905+0000", 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00073871500000000003 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045947+0000", 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1845999999999999e-05 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 34/31 ECSubWrite(tid=3, reqid=client.4568.0:62, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:57.044550+0000", 2024-04-18T23:11:07.815 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.723232454, 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001753697, 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.044550+0000", 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.044550+0000", 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.044552+0000", 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7439999999999999e-06 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.044594+0000", 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1862e-05 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.044620+0000", 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6432e-05 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.044644+0000", 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3989999999999999e-05 2024-04-18T23:11:07.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045469+0000", 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082510099999999996 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045475+0000", 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6620000000000002e-06 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.046279+0000", 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00080401699999999997 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.046304+0000", 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4888999999999998e-05 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=55, reqid=client.4568.0:181, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.759583+0000", 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.0081994700000001, 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003431815, 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759583+0000", 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759583+0000", 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759584+0000", 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.72e-07 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759697+0000", 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113799 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759712+0000", 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4446e-05 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759722+0000", 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0023e-05 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759793+0000", 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1229999999999994e-05 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759803+0000", 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7149999999999993e-06 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.762980+0000", 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0031774210000000002 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.763015+0000", 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4909000000000001e-05 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=56, reqid=client.4568.0:182, at_version=36'14, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.759745+0000", 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.0080379329999998, 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060262479999999997, 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759745+0000", 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759745+0000", 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759745+0000", 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.39e-07 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759815+0000", 2024-04-18T23:11:07.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9621000000000003e-05 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759824+0000", 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8089999999999999e-06 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759832+0000", 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8909999999999995e-06 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760463+0000", 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063172399999999996 2024-04-18T23:11:07.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760470+0000", 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2199999999999997e-06 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.765740+0000", 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052702080000000002 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.765771+0000", 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1335999999999999e-05 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.826 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=57, reqid=client.4568.0:183, at_version=36'15, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.759844+0000", 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 7.0079388920000003, 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059428329999999998, 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759844+0000", 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759844+0000", 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759844+0000", 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7500000000000001e-07 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759856+0000", 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1426999999999999e-05 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759868+0000", 2024-04-18T23:11:07.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2673999999999999e-05 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759875+0000", 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1380000000000001e-06 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761232+0000", 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001356583 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761237+0000", 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1590000000000001e-06 2024-04-18T23:11:07.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.765776+0000", 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004538416 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.765787+0000", 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1061e-05 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.830 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=63, reqid=client.4568.0:183, at_version=0'0, trim_to=0'0, roll_forward_to=36'15))", 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.068067+0000", 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.6997157229999997, 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10038164300000001, 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068067+0000", 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068067+0000", 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068068+0000", 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.35e-06 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068083+0000", 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4695000000000001e-05 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068105+0000", 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1696999999999999e-05 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068119+0000", 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4367e-05 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068210+0000", 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0946999999999995e-05 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068235+0000", 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4499e-05 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168435+0000", 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100200154 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168449+0000", 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3934000000000001e-05 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.bs3 36/31 ECSubRead(tid=64, to_read={3:dd01ba5f:::smithi14230322-23:head=0,4096,0}, subchunks={3:dd01ba5f:::smithi14230322-23:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.068146+0000", 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.6996366759999999, 2024-04-18T23:11:07.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100284543, 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068146+0000", 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068146+0000", 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068147+0000", 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.34e-07 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068154+0000", 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9990000000000002e-06 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068165+0000", 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1968e-05 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068176+0000", 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0018e-05 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168365+0000", 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10018914499999999 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168431+0000", 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5979e-05 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=66, reqid=client.4568.0:189, at_version=36'17, trim_to=0'0, roll_forward_to=36'15))", 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.171444+0000", 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.5963388319999998, 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0024583840000000001, 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171444+0000", 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171444+0000", 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171445+0000", 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.006e-06 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171483+0000", 2024-04-18T23:11:07.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7789e-05 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171505+0000", 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.262e-05 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171514+0000", 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.276e-06 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171571+0000", 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7601e-05 2024-04-18T23:11:07.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171581+0000", 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0108e-05 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173860+0000", 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022790089999999998 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173902+0000", 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1974999999999997e-05 2024-04-18T23:11:07.841 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=67, reqid=client.4568.0:189, at_version=0'0, trim_to=0'0, roll_forward_to=36'17))", 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.470594+0000", 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.297189253, 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10183064999999999, 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470594+0000", 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470594+0000", 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470596+0000", 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0669999999999999e-06 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470621+0000", 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222999999999999e-05 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470688+0000", 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6824999999999998e-05 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470713+0000", 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5474999999999999e-05 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.570999+0000", 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10028558899999999 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.571026+0000", 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7676000000000001e-05 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.572255+0000", 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012286549999999999 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.572424+0000", 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016914 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.845 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 42/31 ECSubWrite(tid=85, reqid=osd.4.0:85, at_version=42'27, trim_to=0'0, roll_forward_to=40'25))", 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.190050447, 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16837443799999999, 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:07.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577733+0000", 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6300000000000001e-07 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577758+0000", 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5099000000000001e-05 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577779+0000", 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0937999999999998e-05 2024-04-18T23:11:07.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577792+0000", 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3227999999999999e-05 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744736+0000", 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.166943963 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744769+0000", 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2629000000000002e-05 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746075+0000", 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001306338 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746107+0000", 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.188e-05 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:07.849 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:07.850 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:07.851 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:11:07.852 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:07.852 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:07.853 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:07.851+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:07.876 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:07.878 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:07.879 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:07.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 'trim stale osdmaps' 2024-04-18T23:11:07.910 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:07.911 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:07.931 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:07.931 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:07.951 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:07.952+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:07.985 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:07.985 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:08.000 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:11:08.031 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:08.031 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:08.049 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:08.049 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:08.051 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:08.053 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:08.054 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:08.054 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 'trim stale osdmaps' 2024-04-18T23:11:08.070 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:08.070 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:08.086 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:08.088+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:08.091 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:08.091 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:08.109 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:08.110 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:08.127 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:08.128 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:08.148 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:08.148 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:08.153 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:08.152+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:08.163 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:08.163 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:08.168 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:08.169 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:08.178 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:11:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:08.212 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:08.213 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:08.215 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:08.215 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:08.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 'trim stale osdmaps' 2024-04-18T23:11:08.254 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:08.252+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:11:08.349 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:08.350 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:08.355 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:08.356+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:08.364 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:11:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:08.415 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:08.416 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:08.416 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 'trim stale osdmaps' 2024-04-18T23:11:08.465 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:08.462+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:08.531 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:08.531 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:08.545 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:11:08.557 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:08.555+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:08.566 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:08.566 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:08.567 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:08.569 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:08.570 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:08.570 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 'trim stale osdmaps' 2024-04-18T23:11:08.593 INFO:tasks.rados.rados.0.smithi142.stdout:114: done (5 left) 2024-04-18T23:11:08.593 INFO:tasks.rados.rados.0.smithi142.stdout:115: copy_from oid 38 from oid 5 current snap is 5 2024-04-18T23:11:08.593 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 14 v 43 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-18T23:11:08.593 INFO:tasks.rados.rados.0.smithi142.stdout:112: finishing write tid 1 to smithi14230322-25 2024-04-18T23:11:08.593 INFO:tasks.rados.rados.0.smithi142.stdout:109: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-18T23:11:08.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T23:11:08.644 INFO:tasks.rados.rados.0.smithi142.stdout:112: finishing write tid 2 to smithi14230322-25 2024-04-18T23:11:08.644 INFO:tasks.rados.rados.0.smithi142.stdout:112: finishing write tid 3 to smithi14230322-25 2024-04-18T23:11:08.644 INFO:tasks.rados.rados.0.smithi142.stdout:113: finishing copy_from racing read to smithi14230322-32 2024-04-18T23:11:08.644 INFO:tasks.rados.rados.0.smithi142.stdout:113: finishing copy_from to smithi14230322-32 2024-04-18T23:11:08.644 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 32 v 23 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-18T23:11:08.644 INFO:tasks.rados.rados.0.smithi142.stdout:112: finishing write tid 4 to smithi14230322-25 2024-04-18T23:11:08.645 INFO:tasks.rados.rados.0.smithi142.stdout:112: oid 25 updating version 0 to 22 2024-04-18T23:11:08.645 INFO:tasks.rados.rados.0.smithi142.stdout:112: oid 25 version 22 is already newer than 21 2024-04-18T23:11:08.645 INFO:tasks.rados.rados.0.smithi142.stdout:112: oid 25 updating version 22 to 23 2024-04-18T23:11:08.645 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 25 v 23 (ObjNum 59 snap 5 seq_num 59) dirty exists 2024-04-18T23:11:08.645 INFO:tasks.rados.rados.0.smithi142.stdout:112: left oid 25 (ObjNum 59 snap 5 seq_num 59) 2024-04-18T23:11:08.645 INFO:tasks.rados.rados.0.smithi142.stdout:111: expect (ObjNum 8 snap 0 seq_num 8) 2024-04-18T23:11:08.652 INFO:tasks.rados.rados.0.smithi142.stdout:109: done (5 left) 2024-04-18T23:11:08.652 INFO:tasks.rados.rados.0.smithi142.stdout:110: done (4 left) 2024-04-18T23:11:08.652 INFO:tasks.rados.rados.0.smithi142.stdout:111: done (3 left) 2024-04-18T23:11:08.653 INFO:tasks.rados.rados.0.smithi142.stdout:112: done (2 left) 2024-04-18T23:11:08.653 INFO:tasks.rados.rados.0.smithi142.stdout:113: done (1 left) 2024-04-18T23:11:08.653 INFO:tasks.rados.rados.0.smithi142.stdout:116: read oid 14 snap 2 2024-04-18T23:11:08.653 INFO:tasks.rados.rados.0.smithi142.stdout:116: expect (ObjNum 13 snap 0 seq_num 13) 2024-04-18T23:11:08.653 INFO:tasks.rados.rados.0.smithi142.stdout:117: snap_create 2024-04-18T23:11:08.658 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:08.656+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:08.667 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:08.667 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:08.681 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:11:08.758 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:08.758 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:08.758 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:08.758 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:08.758 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:08.758 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:08.758 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:08.758 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:08.759 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:08.759 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:08.759 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:08.759 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:08.760 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:08.761 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:08.761 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 'trim stale osdmaps' 2024-04-18T23:11:08.770 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:08.772+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:08.846 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:08.846 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:08.859 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:08.860+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:08.861 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:11:08.905 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:08.906 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:08.908 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:08.908 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:08.909 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 'trim stale osdmaps' 2024-04-18T23:11:08.960 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:08.960+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:09.022 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:09.022 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:09.037 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:0 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:1 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:2 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:abort 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:assert 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:bench [] [] [] [] 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs files list 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:bluefs stats 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:bluestore allocator dump block 2024-04-18T23:11:09.049 INFO:teuthology.orchestra.run.smithi073.stderr:Error EINVAL: invalid command 2024-04-18T23:11:09.052 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:09.052 INFO:tasks.thrashosds.thrasher:Failed to trim stale osdmaps, ignoring 2024-04-18T23:11:09.061 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:09.056+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:09.160 INFO:tasks.rados.rados.0.smithi142.stdout:115: finishing copy_from racing read to smithi14230322-38 2024-04-18T23:11:09.162 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:09.160+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:09.202 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:09.202 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:09.218 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:11:09.261 INFO:tasks.rados.rados.0.smithi142.stdout:116: expect (ObjNum 13 snap 0 seq_num 13) 2024-04-18T23:11:09.262 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:09.264+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:09.280 INFO:tasks.rados.rados.0.smithi142.stdout:115: finishing copy_from to smithi14230322-38 2024-04-18T23:11:09.280 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 38 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-18T23:11:09.339 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:09.340 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:09.353 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:11:09.363 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:09.358+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:09.464 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:09.460+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:09.517 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:09.518 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:09.532 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:11:09.565 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:09.562+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:09.601 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-04-18T23:11:09.605 INFO:tasks.rados.rados.0.smithi142.stdout:115: done (2 left) 2024-04-18T23:11:09.606 INFO:tasks.rados.rados.0.smithi142.stdout:116: done (1 left) 2024-04-18T23:11:09.606 INFO:tasks.rados.rados.0.smithi142.stdout:117: done (0 left) 2024-04-18T23:11:09.606 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 9 current snap is 6 2024-04-18T23:11:09.606 INFO:tasks.rados.rados.0.smithi142.stdout:118: seq_num 60 ranges {147456=638976,786432=376832} 2024-04-18T23:11:09.618 INFO:tasks.rados.rados.0.smithi142.stdout:118: writing smithi14230322-9 from 147456 to 786432 tid 1 2024-04-18T23:11:09.625 INFO:tasks.rados.rados.0.smithi142.stdout:118: writing smithi14230322-9 from 786432 to 1163264 tid 2 2024-04-18T23:11:09.625 INFO:tasks.rados.rados.0.smithi142.stdout:119: rollback oid 3 current snap is 6 2024-04-18T23:11:09.625 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 3 to 2 2024-04-18T23:11:09.625 INFO:tasks.rados.rados.0.smithi142.stdout:120: read oid 25 snap -1 2024-04-18T23:11:09.625 INFO:tasks.rados.rados.0.smithi142.stdout:120: expect (ObjNum 59 snap 5 seq_num 59) 2024-04-18T23:11:09.625 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 2 current snap is 6 2024-04-18T23:11:09.625 INFO:tasks.rados.rados.0.smithi142.stdout:121: seq_num 61 ranges {1130496=655360,1785856=376832} 2024-04-18T23:11:09.653 INFO:tasks.rados.rados.0.smithi142.stdout:121: writing smithi14230322-2 from 1130496 to 1785856 tid 1 2024-04-18T23:11:09.655 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:09.655 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:09.659 INFO:tasks.rados.rados.0.smithi142.stdout:121: writing smithi14230322-2 from 1785856 to 2162688 tid 2 2024-04-18T23:11:09.659 INFO:tasks.rados.rados.0.smithi142.stdout:122: snap_create 2024-04-18T23:11:09.665 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:09.664+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:09.669 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:11:09.766 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:09.762+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:09.838 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:09.838 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:09.853 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:11:09.867 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:09.864+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:11:09.947 INFO:tasks.rados.rados.0.smithi142.stdout:121: finishing write tid 1 to smithi14230322-2 2024-04-18T23:11:09.968 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:09.968+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:09.987 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:09.987 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:10.001 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:11:10.046 INFO:tasks.rados.rados.0.smithi142.stdout:120: expect (ObjNum 59 snap 5 seq_num 59) 2024-04-18T23:11:10.069 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:10.068+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:10.121 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:10.122 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:10.136 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:11:10.169 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:10.168+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:10.184 INFO:tasks.rados.rados.0.smithi142.stdout:121: finishing write tid 2 to smithi14230322-2 2024-04-18T23:11:10.184 INFO:tasks.rados.rados.0.smithi142.stdout:121: finishing write tid 3 to smithi14230322-2 2024-04-18T23:11:10.184 INFO:tasks.rados.rados.0.smithi142.stdout:118: finishing write tid 1 to smithi14230322-9 2024-04-18T23:11:10.187 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:10.252 INFO:tasks.rados.rados.0.smithi142.stdout:118: finishing write tid 2 to smithi14230322-9 2024-04-18T23:11:10.252 INFO:tasks.rados.rados.0.smithi142.stdout:118: finishing write tid 3 to smithi14230322-9 2024-04-18T23:11:10.258 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:10.258 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:10.270 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:10.268+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:10.272 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:11:10.333 INFO:tasks.rados.rados.0.smithi142.stdout:119: finishing rollback tid 0 to smithi14230322-3 2024-04-18T23:11:10.333 INFO:tasks.rados.rados.0.smithi142.stdout:119: finishing rollback tid 1 to smithi14230322-3 2024-04-18T23:11:10.333 INFO:tasks.rados.rados.0.smithi142.stdout:119: finishing rollback tid 2 to smithi14230322-3 2024-04-18T23:11:10.333 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 3 v 29 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-18T23:11:10.341 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:10.362 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:10.362 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:10.370 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:10.366+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:10.382 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:10.382 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:10.392 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:10.393 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:10.403 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:10.403 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:10.407 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:11:10.421 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:10.421 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:10.435 INFO:tasks.rados.rados.0.smithi142.stdout:121: finishing write tid 4 to smithi14230322-2 2024-04-18T23:11:10.435 INFO:tasks.rados.rados.0.smithi142.stdout:121: oid 2 updating version 0 to 26 2024-04-18T23:11:10.435 INFO:tasks.rados.rados.0.smithi142.stdout:121: oid 2 updating version 26 to 27 2024-04-18T23:11:10.436 INFO:tasks.rados.rados.0.smithi142.stdout:121: oid 2 version 27 is already newer than 25 2024-04-18T23:11:10.436 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 2 v 27 (ObjNum 61 snap 6 seq_num 61) dirty exists 2024-04-18T23:11:10.436 INFO:tasks.rados.rados.0.smithi142.stdout:121: left oid 2 (ObjNum 61 snap 6 seq_num 61) 2024-04-18T23:11:10.443 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:10.455 INFO:tasks.rados.rados.0.smithi142.stdout:118: finishing write tid 4 to smithi14230322-9 2024-04-18T23:11:10.455 INFO:tasks.rados.rados.0.smithi142.stdout:118: oid 9 updating version 0 to 19 2024-04-18T23:11:10.455 INFO:tasks.rados.rados.0.smithi142.stdout:118: oid 9 version 19 is already newer than 18 2024-04-18T23:11:10.455 INFO:tasks.rados.rados.0.smithi142.stdout:118: oid 9 updating version 19 to 20 2024-04-18T23:11:10.455 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 9 v 20 (ObjNum 60 snap 6 seq_num 60) dirty exists 2024-04-18T23:11:10.455 INFO:tasks.rados.rados.0.smithi142.stdout:118: left oid 9 (ObjNum 60 snap 6 seq_num 60) 2024-04-18T23:11:10.463 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:10.463 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:10.471 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:10.470+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:10.485 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:10.485 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:10.502 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:10.502 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:10.524 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:10.524 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:10.528 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:10.528 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:10.528 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:10.528 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:10.528 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:10.542 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:11:10.544 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:10.544 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:10.567 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:10.567 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:10.572 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:10.568+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:10.583 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:10.607 INFO:tasks.rados.rados.0.smithi142.stdout:118: done (4 left) 2024-04-18T23:11:10.607 INFO:tasks.rados.rados.0.smithi142.stdout:119: done (3 left) 2024-04-18T23:11:10.607 INFO:tasks.rados.rados.0.smithi142.stdout:120: done (2 left) 2024-04-18T23:11:10.607 INFO:tasks.rados.rados.0.smithi142.stdout:121: done (1 left) 2024-04-18T23:11:10.607 INFO:tasks.rados.rados.0.smithi142.stdout:122: done (0 left) 2024-04-18T23:11:10.607 INFO:tasks.rados.rados.0.smithi142.stdout:123: read oid 48 snap 5 2024-04-18T23:11:10.608 INFO:tasks.rados.rados.0.smithi142.stdout:123: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:10.608 INFO:tasks.rados.rados.0.smithi142.stdout:124: read oid 49 snap 6 2024-04-18T23:11:10.608 INFO:tasks.rados.rados.0.smithi142.stdout:124: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-18T23:11:10.608 INFO:tasks.rados.rados.0.smithi142.stdout:125: delete oid 44 current snap is 7 2024-04-18T23:11:10.608 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:10.608 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:10.673 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:10.668+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.es5 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.538001+0000", 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.174066837, 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102078829, 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.713 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.538001+0000", 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.538001+0000", 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.538003+0000", 2024-04-18T23:11:10.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4500000000000001e-06 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.538046+0000", 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3625000000000003e-05 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.538080+0000", 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3348e-05 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.638185+0000", 2024-04-18T23:11:10.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100105509 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.638304+0000", 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000118774 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.638322+0000", 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8416000000000001e-05 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.639985+0000", 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016624890000000001 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.640080+0000", 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5217999999999995e-05 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=158, reqid=client.4568.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.084339441999999, 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203058724, 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627730+0000", 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.606e-06 2024-04-18T23:11:10.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627841+0000", 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110692 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627889+0000", 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8211000000000003e-05 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627919+0000", 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9961999999999999e-05 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629700+0000", 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001780963 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629730+0000", 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0082000000000001e-05 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830736+0000", 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201005607 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830787+0000", 2024-04-18T23:11:10.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1601000000000003e-05 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=159, reqid=client.4568.0:25, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.084065628999999, 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20284285199999999, 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628004+0000", 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0270000000000001e-06 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628137+0000", 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013336000000000001 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628197+0000", 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0504000000000002e-05 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628221+0000", 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3037999999999999e-05 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728171+0000", 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099950707999999999 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728190+0000", 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8709e-05 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830795+0000", 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102605138 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830845+0000", 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0368000000000003e-05 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=160, reqid=client.4568.0:26, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.083810908, 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20264686400000001, 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.725 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628258+0000", 2024-04-18T23:11:10.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628292+0000", 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4054999999999998e-05 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628317+0000", 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5035999999999999e-05 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628338+0000", 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1058999999999999e-05 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728828+0000", 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100489521 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728846+0000", 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8419000000000001e-05 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830850+0000", 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10200397 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830904+0000", 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3844000000000002e-05 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=161, reqid=client.4568.0:31, at_version=34'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.083676863999999, 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202594633, 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.729 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628392+0000", 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6000000000000004e-07 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628541+0000", 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149386 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628572+0000", 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0975000000000001e-05 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628594+0000", 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1608999999999999e-05 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829299+0000", 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20070527499999999 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829314+0000", 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4718999999999999e-05 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830915+0000", 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001601412 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830986+0000", 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0697000000000004e-05 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=162, reqid=client.4568.0:32, at_version=34'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.635797+0000", 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.076271109, 2024-04-18T23:11:10.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.195800061, 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635797+0000", 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635797+0000", 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635799+0000", 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9039999999999999e-06 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635849+0000", 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0281000000000002e-05 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635897+0000", 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7416999999999997e-05 2024-04-18T23:11:10.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635923+0000", 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6486e-05 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830004+0000", 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19408114400000001 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830019+0000", 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4533e-05 2024-04-18T23:11:10.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.831518+0000", 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014992300000000001 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.831597+0000", 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9066000000000005e-05 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 34/31 ECSubWrite(tid=6, reqid=client.4568.0:17, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.733922+0000", 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.978146109000001, 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10153214300000001, 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.733922+0000", 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.733922+0000", 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.733924+0000", 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.857e-06 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.733953+0000", 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9020999999999999e-05 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.733988+0000", 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5332e-05 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.734041+0000", 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2564e-05 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.834264+0000", 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100222967 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.834278+0000", 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4093e-05 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.835376+0000", 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010985299999999999 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.835454+0000", 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7779000000000002e-05 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.741 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs4 34/31 ECSubWrite(tid=12, reqid=client.4568.0:20, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.836302+0000", 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.875766690000001, 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101962618, 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.836302+0000", 2024-04-18T23:11:10.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.836302+0000", 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.836302+0000", 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0999999999999997e-07 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.836348+0000", 2024-04-18T23:11:10.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5494000000000002e-05 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.836368+0000", 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0299e-05 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.936467+0000", 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100098626 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.936636+0000", 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000169295 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.936663+0000", 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6478e-05 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.938165+0000", 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001502527 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.938264+0000", 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8988999999999997e-05 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.745 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 36/31 ECSubWrite(tid=54, reqid=client.4568.0:188, at_version=36'19, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.762269+0000", 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9497994579999993, 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103903021, 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762269+0000", 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762269+0000", 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762269+0000", 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762379+0000", 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110065 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762407+0000", 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7515e-05 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762427+0000", 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0392999999999999e-05 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762564+0000", 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013708399999999999 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762583+0000", 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8868999999999998e-05 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866094+0000", 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103510981 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866172+0000", 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7459999999999994e-05 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 36/31 ECSubWrite(tid=58, reqid=client.4568.0:196, at_version=36'21, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.763024+0000", 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9490437739999997, 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103273062, 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.750 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763024+0000", 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763024+0000", 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763025+0000", 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2e-07 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763067+0000", 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2135000000000002e-05 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763078+0000", 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1362000000000001e-05 2024-04-18T23:11:10.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763088+0000", 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1900000000000001e-06 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763717+0000", 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000629179 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763741+0000", 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4572000000000001e-05 2024-04-18T23:11:10.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866222+0000", 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102480577 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866297+0000", 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5626999999999999e-05 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 36/31 ECSubWrite(tid=59, reqid=client.4568.0:197, at_version=36'22, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.763568+0000", 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9485003849999991, 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103433918, 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763568+0000", 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763568+0000", 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763569+0000", 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.06e-06 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763596+0000", 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7013e-05 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763627+0000", 2024-04-18T23:11:10.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1531e-05 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763639+0000", 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.865607+0000", 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101967455 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.865622+0000", 2024-04-18T23:11:10.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4973e-05 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866932+0000", 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013096029999999999 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.867002+0000", 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0222999999999995e-05 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 36/31 ECSubWrite(tid=60, reqid=client.4568.0:198, at_version=36'23, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.764316+0000", 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.9477517780000007, 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103564976, 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764316+0000", 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764316+0000", 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764318+0000", 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4169999999999999e-06 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764357+0000", 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9499000000000002e-05 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764418+0000", 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0603999999999998e-05 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764439+0000", 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1239000000000001e-05 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866401+0000", 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101961882 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866431+0000", 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9578999999999999e-05 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.867805+0000", 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013745059999999999 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.867881+0000", 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6249999999999997e-05 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.763 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 36/31 ECSubWrite(tid=9, reqid=client.4568.0:176, at_version=36'8, trim_to=0'0, roll_forward_to=34'4))", 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.956436+0000", 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 9.7556318229999999, 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10256767999999999, 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956436+0000", 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956436+0000", 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956438+0000", 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956564+0000", 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000126317 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956589+0000", 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5148999999999999e-05 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956613+0000", 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3731999999999999e-05 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056863+0000", 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10025031700000001 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056887+0000", 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3552000000000001e-05 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.058940+0000", 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020530589999999999 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.059004+0000", 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4073999999999997e-05 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs4 42/31 ECSubWrite(tid=85, reqid=osd.4.0:85, at_version=42'27, trim_to=0'0, roll_forward_to=40'25))", 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.580086+0000", 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.1319817900000002, 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10184555200000001, 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.580086+0000", 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.580086+0000", 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.580087+0000", 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4000000000000001e-07 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.580152+0000", 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4684999999999995e-05 2024-04-18T23:11:10.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.580184+0000", 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2141999999999998e-05 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.580215+0000", 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0982000000000003e-05 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.680523+0000", 2024-04-18T23:11:10.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100308613 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.680539+0000", 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5216e-05 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681843+0000", 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013041839999999999 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681932+0000", 2024-04-18T23:11:10.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9090000000000003e-05 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.es5 42/31 ECSubWrite(tid=24, reqid=osd.2.0:24, at_version=42'14, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.596307+0000", 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.1157616450000001, 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10188396299999999, 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.596307+0000", 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.596307+0000", 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.596307+0000", 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4999999999999998e-07 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.596350+0000", 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2664999999999999e-05 2024-04-18T23:11:10.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.596379+0000", 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8912999999999999e-05 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.596395+0000", 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6748999999999998e-05 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.696601+0000", 2024-04-18T23:11:10.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100206113 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.696617+0000", 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5113000000000001e-05 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.698120+0000", 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001503597 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.698191+0000", 2024-04-18T23:11:10.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0363000000000004e-05 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.es5 42/31 ECSubWrite(tid=25, reqid=osd.2.0:24, at_version=0'0, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.799564+0000", 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.9125046650000002, 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18457921799999999, 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799564+0000", 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799564+0000", 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799565+0000", 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799594+0000", 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8221999999999998e-05 2024-04-18T23:11:10.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799640+0000", 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6749000000000003e-05 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799665+0000", 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4712e-05 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.883580+0000", 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.083914739000000002 2024-04-18T23:11:10.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.883592+0000", 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2595000000000001e-05 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.984082+0000", 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100489984 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.984143+0000", 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0404e-05 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.779 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 42/31 ECSubWrite(tid=89, reqid=client.4568.0:279, at_version=42'41, trim_to=0'0, roll_forward_to=42'39))", 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.094462+0000", 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.6176065299999998, 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10436654099999999, 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.094462+0000", 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.094462+0000", 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.094464+0000", 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0200000000000001e-06 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.095463+0000", 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009991100000000001 2024-04-18T23:11:10.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.095494+0000", 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0691999999999997e-05 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.195636+0000", 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100142097 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.195800+0000", 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000164057 2024-04-18T23:11:10.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.195815+0000", 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4972e-05 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.198757+0000", 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029427590000000001 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.198828+0000", 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0834000000000007e-05 2024-04-18T23:11:10.783 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s5 42/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=42'12))", 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.669347+0000", 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.0427213790000001, 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20082834299999999, 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669347+0000", 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669347+0000", 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669348+0000", 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7400000000000002e-07 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669371+0000", 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3332e-05 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669458+0000", 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6646999999999999e-05 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669484+0000", 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6706000000000001e-05 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.769505+0000", 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100021191 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.769528+0000", 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2132000000000001e-05 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.870118+0000", 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100590156 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.870175+0000", 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7404999999999997e-05 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s6 43/31 ECSubWrite(tid=33, reqid=osd.15.0:33, at_version=43'19, trim_to=0'0, roll_forward_to=42'17))", 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.680855+0000", 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 3.031213111, 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101834644, 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.788 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.680855+0000", 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.680855+0000", 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.680855+0000", 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.84e-07 2024-04-18T23:11:10.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.680911+0000", 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5250999999999997e-05 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.680934+0000", 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3555999999999999e-05 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.680950+0000", 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5339e-05 2024-04-18T23:11:10.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.781231+0000", 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100281328 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.781256+0000", 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5191999999999999e-05 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.782616+0000", 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013599370000000001 2024-04-18T23:11:10.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.782690+0000", 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3657000000000003e-05 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s6 43/31 ECSubWrite(tid=36, reqid=client.4568.0:291, at_version=43'23, trim_to=0'0, roll_forward_to=42'17))", 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.792105+0000", 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.9199629520000001, 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103170358, 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:10.792 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792105+0000", 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792105+0000", 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792112+0000", 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8619999999999997e-06 2024-04-18T23:11:10.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792145+0000", 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2571999999999999e-05 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792172+0000", 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7253000000000001e-05 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.892267+0000", 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100095301 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.892409+0000", 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141695 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.892441+0000", 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2016e-05 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.895187+0000", 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027456820000000002 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:10.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.895276+0000", 2024-04-18T23:11:10.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8976999999999999e-05 2024-04-18T23:11:10.796 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.796 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.796 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.796 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:10.796 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:10.796 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:10.797 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:10.798 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:10.799 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:11:10.799 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:10.794+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:10.800 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:10.899 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:10.894+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:10.969 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:10.970 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:10.970 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:11:10.970 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:10.983 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:11:10.999 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:10.994+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:11.058 INFO:tasks.rados.rados.0.smithi142.stdout:124: expect (ObjNum 48 snap 0 seq_num 48) 2024-04-18T23:11:11.100 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:11.098+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:11.107 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:11.107 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:11.107 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:11.107 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:11.107 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:11.117 INFO:tasks.rados.rados.0.smithi142.stdout:123: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:11.121 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:11:11.201 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:11.200+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:11.246 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:11.246 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:11.246 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:11.246 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:11.246 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.246 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.1s4 33/31 ECSubWrite(tid=0, reqid=client.4568.0:1, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.524912+0000", 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.720897382, 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021653079999999999, 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.524912+0000", 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.247 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.524912+0000", 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.524913+0000", 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0130000000000001e-06 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.525019+0000", 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000106025 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.248 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.525046+0000", 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6738999999999999e-05 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.525070+0000", 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4272000000000001e-05 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.525292+0000", 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022236599999999999 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.249 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.525320+0000", 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8357999999999999e-05 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.526979+0000", 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016580130000000001 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.527077+0000", 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8522000000000003e-05 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.250 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 34/31 ECSubWrite(tid=0, reqid=client.4568.0:7, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.528298+0000", 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.71751108, 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003543147, 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528298+0000", 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.251 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528298+0000", 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528299+0000", 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9600000000000008e-07 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528473+0000", 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017360800000000001 2024-04-18T23:11:11.252 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528507+0000", 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3971999999999999e-05 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528538+0000", 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1684999999999999e-05 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.529242+0000", 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070414900000000005 2024-04-18T23:11:11.253 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.529255+0000", 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2850000000000001e-05 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.531741+0000", 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024855620000000002 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.531841+0000", 2024-04-18T23:11:11.254 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100325 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.1s4 33/31 ECSubWrite(tid=3, reqid=client.4568.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.528321+0000", 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.717487982, 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002443335, 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.255 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528321+0000", 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528321+0000", 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528322+0000", 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9999999999999995e-07 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.256 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528389+0000", 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6414000000000005e-05 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528421+0000", 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2617999999999998e-05 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528453+0000", 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1968999999999998e-05 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.257 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528695+0000", 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024193399999999999 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528720+0000", 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4556e-05 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.530676+0000", 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001956312 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.258 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.530764+0000", 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8531999999999999e-05 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 34/31 ECSubWrite(tid=0, reqid=client.4568.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.535047+0000", 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.710761972, 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099095934999999996, 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.259 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535047+0000", 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535047+0000", 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535048+0000", 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.96e-07 2024-04-18T23:11:11.260 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535249+0000", 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020098699999999999 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535280+0000", 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1301999999999999e-05 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535304+0000", 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3918999999999999e-05 2024-04-18T23:11:11.261 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.632426+0000", 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.097122690999999997 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.632441+0000", 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4273e-05 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.634033+0000", 2024-04-18T23:11:11.262 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001592729 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.634143+0000", 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109638 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:29, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.636644+0000", 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.609165455999999, 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10234691999999999, 2024-04-18T23:11:11.263 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636644+0000", 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636644+0000", 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.264 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636646+0000", 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.103e-06 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636689+0000", 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3464999999999998e-05 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636729+0000", 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0005000000000001e-05 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.265 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636763+0000", 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3417999999999998e-05 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.737095+0000", 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100332765 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.737122+0000", 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6355000000000001e-05 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.266 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.738921+0000", 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017988730000000001 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.738991+0000", 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9936000000000003e-05 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.1s4 34/31 ECSubWrite(tid=10, reqid=client.4568.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.825267+0000", 2024-04-18T23:11:11.267 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.420542076, 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10131675399999999, 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825267+0000", 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825267+0000", 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.268 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825269+0000", 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.844e-06 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825298+0000", 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9065e-05 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825331+0000", 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3062000000000003e-05 2024-04-18T23:11:11.269 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.925491+0000", 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100160233 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.925692+0000", 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000201263 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.925708+0000", 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5913000000000002e-05 2024-04-18T23:11:11.270 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.926517+0000", 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080818100000000005 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.926584+0000", 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7193000000000005e-05 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.271 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 34/31 ECSubWrite(tid=0, reqid=client.4568.0:58, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.944422+0000", 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.301387550999999, 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023273500000000002, 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.944422+0000", 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.272 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.944422+0000", 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.944424+0000", 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.097e-06 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.944711+0000", 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028690200000000001 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.273 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.944741+0000", 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0849000000000002e-05 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.944758+0000", 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6804999999999999e-05 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.274 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.944928+0000", 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016953899999999999 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.944956+0000", 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8309e-05 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.946694+0000", 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017380379999999999 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.275 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.946749+0000", 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4811000000000001e-05 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:59, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.945228+0000", 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.300580913999999, 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022239820000000002, 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.276 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.945228+0000", 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.945228+0000", 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.945229+0000", 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.28e-06 2024-04-18T23:11:11.277 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.945257+0000", 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7729000000000002e-05 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.945286+0000", 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8742e-05 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.945319+0000", 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2855000000000003e-05 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.278 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.946174+0000", 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085566799999999997 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.946201+0000", 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6631999999999998e-05 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.947383+0000", 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011818390000000001 2024-04-18T23:11:11.279 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.947452+0000", 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9237000000000001e-05 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 34/31 ECSubWrite(tid=2, reqid=client.4568.0:61, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:57.046581+0000", 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.199227717999999, 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024022190000000001, 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.280 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.046581+0000", 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.046581+0000", 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.281 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.046583+0000", 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.04e-06 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.046916+0000", 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00033305100000000001 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.046953+0000", 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6817999999999999e-05 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.282 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.046979+0000", 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5395999999999999e-05 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047158+0000", 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017899800000000001 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047184+0000", 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6055e-05 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.283 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.048905+0000", 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001720905 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.048984+0000", 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8955999999999994e-05 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 34/31 ECSubWrite(tid=3, reqid=client.4568.0:62, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:57.047096+0000", 2024-04-18T23:11:11.284 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.198712912, 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035305250000000001, 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047096+0000", 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047096+0000", 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.285 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047097+0000", 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2899999999999999e-06 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047140+0000", 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2166999999999998e-05 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.286 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047166+0000", 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5939e-05 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047187+0000", 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1866e-05 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.048317+0000", 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011297830000000001 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.048344+0000", 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6911999999999999e-05 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.050579+0000", 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002234556 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.050627+0000", 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8012e-05 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.288 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 36/31 ECSubRead(tid=22, to_read={3:b210783d:::smithi14230322-30:head=0,122880,0}, subchunks={3:b210783d:::smithi14230322-30:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.545989+0000", 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.699819871999999, 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10058726900000001, 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.545989+0000", 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.545989+0000", 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.545990+0000", 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4299999999999999e-07 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546004+0000", 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4523e-05 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546036+0000", 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1429e-05 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546068+0000", 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2654999999999998e-05 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646370+0000", 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100302105 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646577+0000", 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000206214 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.292 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 36/31 ECSubRead(tid=23, to_read={3:b210783d:::smithi14230322-30:head=0,122880,0}, subchunks={3:b210783d:::smithi14230322-30:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.546648+0000", 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.699161531, 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100272213, 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546648+0000", 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546648+0000", 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546649+0000", 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6139999999999999e-06 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546665+0000", 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6016e-05 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546700+0000", 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4857999999999997e-05 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546730+0000", 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0232000000000001e-05 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646687+0000", 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099956749999999997 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646920+0000", 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023274299999999999 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 36/31 ECSubRead(tid=24, to_read={3:b210783d:::smithi14230322-30:head=0,122880,0}, subchunks={3:b210783d:::smithi14230322-30:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.548897+0000", 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.696911989, 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098321620999999998, 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548897+0000", 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548897+0000", 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548899+0000", 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.607e-06 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548915+0000", 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5985999999999999e-05 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548950+0000", 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4972000000000003e-05 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548999+0000", 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9740999999999997e-05 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.647040+0000", 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098040085999999999 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.647219+0000", 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017922899999999999 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=25, reqid=client.4568.0:185, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:11:11.299 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.595725808999999, 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103575165, 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:11:11.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650085+0000", 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8929999999999999e-06 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650164+0000", 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8852999999999999e-05 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650193+0000", 2024-04-18T23:11:11.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9352e-05 2024-04-18T23:11:11.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750332+0000", 2024-04-18T23:11:11.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100138779 2024-04-18T23:11:11.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750511+0000", 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000178679 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750538+0000", 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328999999999999e-05 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753559+0000", 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030212080000000001 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753658+0000", 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9072000000000003e-05 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=26, reqid=client.4568.0:186, at_version=36'14, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.750398+0000", 2024-04-18T23:11:11.304 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.495411258000001, 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038214529999999998, 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750398+0000", 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750398+0000", 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750399+0000", 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0899999999999999e-06 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750442+0000", 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3115000000000003e-05 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750469+0000", 2024-04-18T23:11:11.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6579000000000001e-05 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750490+0000", 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1075000000000001e-05 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752513+0000", 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020229050000000002 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752538+0000", 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5868999999999999e-05 2024-04-18T23:11:11.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.754153+0000", 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016146260000000001 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.754219+0000", 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6193999999999997e-05 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.308 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=27, reqid=client.4568.0:187, at_version=36'15, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.752447+0000", 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.493361808, 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022500020000000001, 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752447+0000", 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752447+0000", 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752449+0000", 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.643e-06 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752497+0000", 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8065000000000001e-05 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752532+0000", 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5061999999999997e-05 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752595+0000", 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3204e-05 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753273+0000", 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00067803000000000004 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753288+0000", 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4277000000000001e-05 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.754651+0000", 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001363146 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.754697+0000", 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6575e-05 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=28, reqid=client.4568.0:187, at_version=0'0, trim_to=0'0, roll_forward_to=36'15))", 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:01.056575+0000", 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.189234095, 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001975002, 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056575+0000", 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056575+0000", 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056577+0000", 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6700000000000001e-06 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056627+0000", 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0518e-05 2024-04-18T23:11:11.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056671+0000", 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3421000000000003e-05 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056706+0000", 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5281999999999999e-05 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056893+0000", 2024-04-18T23:11:11.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000187345 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056922+0000", 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9224999999999999e-05 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.058467+0000", 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015445509999999999 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.058550+0000", 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2990000000000003e-05 2024-04-18T23:11:11.316 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=29, reqid=client.4568.0:202, at_version=36'17, trim_to=0'0, roll_forward_to=36'15))", 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:01.073962+0000", 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.17184686, 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10286168, 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.317 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073962+0000", 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073962+0000", 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073964+0000", 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6300000000000001e-06 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.074019+0000", 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5228e-05 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.074049+0000", 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0009e-05 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.174167+0000", 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100118289 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.174350+0000", 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018256199999999999 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.174375+0000", 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5023000000000001e-05 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.176733+0000", 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023585030000000001 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.176824+0000", 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0435999999999999e-05 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 42/31 ECSubWrite(tid=61, reqid=osd.14.0:61, at_version=42'25, trim_to=0'0, roll_forward_to=40'23))", 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.960600+0000", 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.2852092190000004, 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021049939999999998, 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.321 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960600+0000", 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960600+0000", 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960602+0000", 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0030000000000001e-06 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960652+0000", 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0228000000000001e-05 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960682+0000", 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9563000000000001e-05 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960699+0000", 2024-04-18T23:11:11.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6976000000000001e-05 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960878+0000", 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017908299999999999 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960894+0000", 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6096e-05 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.962622+0000", 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001727728 2024-04-18T23:11:11.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.962705+0000", 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3317000000000002e-05 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 46/31 ECSubRead(tid=38, to_read={3:ba4438ef:::smithi14230322-48:head=0,8192,0}, subchunks={3:ba4438ef:::smithi14230322-48:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:10.714783+0000", 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.5310262, 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099622317000000002, 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.325 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714783+0000", 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714783+0000", 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714784+0000", 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5799999999999999e-07 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714790+0000", 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.976e-06 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714815+0000", 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5474e-05 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714829+0000", 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3957999999999999e-05 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.814241+0000", 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099411553999999999 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.814405+0000", 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000164697 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.328 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.329 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.329 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.329 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:11.330 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:11:11.331 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:11.330+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:11.423 INFO:tasks.rados.rados.0.smithi142.stdout:123: done (2 left) 2024-04-18T23:11:11.423 INFO:tasks.rados.rados.0.smithi142.stdout:124: done (1 left) 2024-04-18T23:11:11.423 INFO:tasks.rados.rados.0.smithi142.stdout:125: done (0 left) 2024-04-18T23:11:11.423 INFO:tasks.rados.rados.0.smithi142.stdout:126: rollback oid 49 current snap is 7 2024-04-18T23:11:11.423 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 49 to 5 2024-04-18T23:11:11.423 INFO:tasks.rados.rados.0.smithi142.stdout:127: read oid 27 snap -1 2024-04-18T23:11:11.423 INFO:tasks.rados.rados.0.smithi142.stdout:127: expect (ObjNum 26 snap 0 seq_num 26) 2024-04-18T23:11:11.423 INFO:tasks.rados.rados.0.smithi142.stdout:128: setattr oid 41 current snap is 7 2024-04-18T23:11:11.425 INFO:tasks.rados.rados.0.smithi142.stdout:129: delete oid 28 current snap is 7 2024-04-18T23:11:11.431 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:11.432+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:11.506 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:11.506 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:11.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.506 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.9s0 46/31 ECSubReadReply(tid=78, attrs_read=0))", 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.428425+0000", 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.080085886999999994, 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.080126269, 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "no flag points reached", 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428425+0000", 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428425+0000", 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428425+0000", 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428425+0000", 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428434+0000", 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4150000000000005e-06 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.9s0 46/31 ECSubReadReply(tid=78, attrs_read=0))", 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.428505+0000", 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.080005193000000002, 2024-04-18T23:11:11.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.080115959, 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428505+0000", 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428505+0000", 2024-04-18T23:11:11.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428506+0000", 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.96e-07 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428685+0000", 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000179583 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428703+0000", 2024-04-18T23:11:11.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7311000000000001e-05 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428711+0000", 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6780000000000006e-06 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.9s0 46/31 ECSubReadReply(tid=78, attrs_read=0))", 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.428530+0000", 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.079980565000000003, 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.080162345999999995, 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.512 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428530+0000", 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428530+0000", 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428530+0000", 2024-04-18T23:11:11.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0400000000000002e-07 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428808+0000", 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027786299999999999 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428822+0000", 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3237999999999999e-05 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428829+0000", 2024-04-18T23:11:11.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9449999999999993e-06 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:328 3.9s0 3:906aad0a:::smithi14230322-27:head [read 0~442368,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.426352+0000", 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.082159151999999999, 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.082429649999999993, 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.515 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 328 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426352+0000", 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426352+0000", 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426352+0000", 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1299999999999999e-07 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426357+0000", 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8400000000000002e-06 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426369+0000", 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2228999999999999e-05 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426378+0000", 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0529999999999996e-06 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426934+0000", 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00055593899999999998 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.427009+0000", 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4598999999999995e-05 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.518 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:329 3.9s0 3:906aad0a:::smithi14230322-27:head [read 0~442368,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.426395+0000", 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.082115323000000004, 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.082495033999999995, 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 329 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426395+0000", 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426395+0000", 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426396+0000", 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3799999999999996e-07 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426401+0000", 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6909999999999996e-06 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426411+0000", 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0369e-05 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.426420+0000", 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6070000000000008e-06 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.427317+0000", 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089724599999999998 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.427379+0000", 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2188999999999997e-05 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.9s0 46/31 ECSubReadReply(tid=77, attrs_read=0))", 2024-04-18T23:11:11.522 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.428119+0000", 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.080391643999999998, 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.080870007999999993, 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428119+0000", 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428119+0000", 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428119+0000", 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1e-07 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428389+0000", 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026976099999999998 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428395+0000", 2024-04-18T23:11:11.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6239999999999999e-06 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.428398+0000", 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6030000000000001e-06 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 6 2024-04-18T23:11:11.525 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:11.526 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:11:11.527 INFO:tasks.rados.rados.0.smithi142.stdout:127: expect (ObjNum 26 snap 0 seq_num 26) 2024-04-18T23:11:11.531 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:11.528+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:11.624 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T23:11:11.631 INFO:tasks.rados.rados.0.smithi142.stdout:127: done (3 left) 2024-04-18T23:11:11.631 INFO:tasks.rados.rados.0.smithi142.stdout:129: done (2 left) 2024-04-18T23:11:11.631 INFO:tasks.rados.rados.0.smithi142.stdout:130: snap_remove snap 6 2024-04-18T23:11:11.654 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:11.654 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:11.654 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:11.668 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:11:11.669 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:11.666+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:11.733 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:11.730+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:9 3.9s0 3:9e439d70:::smithi14230322-3:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.517656+0000", 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.277949512999999, 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.32292617000000001, 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.796 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 9 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.517656+0000", 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.517656+0000", 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.517658+0000", 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3e-06 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.798 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.517679+0000", 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1242000000000001e-05 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.517698+0000", 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9442000000000001e-05 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.517715+0000", 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6195999999999999e-05 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.799 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.518313+0000", 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00059801099999999996 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.518328+0000", 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5449999999999999e-05 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.531711+0000", 2024-04-18T23:11:11.800 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.013383291 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.531941+0000", 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022987499999999999 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.736879+0000", 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.204938122 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.737091+0000", 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000211413 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.801 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.840583+0000", 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10349182799999999 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:46 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~262144,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.633185+0000", 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.162420655, 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40668528300000001, 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:11.802 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 46 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633185+0000", 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633185+0000", 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.803 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633187+0000", 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2170000000000001e-06 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633632+0000", 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00044460099999999998 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633681+0000", 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8767999999999998e-05 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.804 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633707+0000", 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6075e-05 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633834+0000", 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012730400000000001 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.634058+0000", 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022390600000000001 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.805 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636856+0000", 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027983690000000002 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.638158+0000", 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001301864 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.039713+0000", 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40155475000000002 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.806 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.039871+0000", 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000157429 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:47 3.0s0 3:063b0705:::smithi14230322-16:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.633749+0000", 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.161856586000001, 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.406803621, 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.807 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 47 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633749+0000", 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633749+0000", 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.808 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633751+0000", 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8369999999999999e-06 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633760+0000", 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9199999999999993e-06 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.633781+0000", 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0528999999999999e-05 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.809 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.733876+0000", 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100095746 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.734115+0000", 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023875799999999999 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.734297+0000", 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018187100000000001 2024-04-18T23:11:11.810 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.735268+0000", 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00097088299999999997 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.738649+0000", 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0033812249999999999 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.040283+0000", 2024-04-18T23:11:11.811 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30163390899999998 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.040553+0000", 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026994299999999997 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:206 3.9s0 3:906aad0a:::smithi14230322-27:head [read 0~442368,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.646646+0000", 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.148959481, 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31021462799999999, 2024-04-18T23:11:11.812 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 206 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646646+0000", 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.813 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646646+0000", 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646648+0000", 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.297e-06 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646656+0000", 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8800000000000008e-06 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.814 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646676+0000", 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0152e-05 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646702+0000", 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6149e-05 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646946+0000", 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024371899999999999 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.815 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646960+0000", 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453e-05 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752022+0000", 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105062237 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752259+0000", 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023631999999999999 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.816 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956861+0000", 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20460242100000001 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:214 3.9s0 3:9e84ac80:::smithi14230322-40:head [rollback 3] snapc 3=[3,2] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:01.544464+0000", 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.251141684, 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30662102800000002, 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.817 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 214 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.544464+0000", 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.818 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.544464+0000", 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.544467+0000", 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7499999999999999e-06 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.544481+0000", 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4439e-05 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.819 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.544515+0000", 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3748999999999998e-05 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.544532+0000", 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7209000000000001e-05 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.544659+0000", 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012705099999999999 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.820 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.544672+0000", 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2153e-05 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.647979+0000", 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10330726 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.648133+0000", 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015384700000000001 2024-04-18T23:11:11.821 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.851085+0000", 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20295257 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:273 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.2140172439999999, 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.781474428, 2024-04-18T23:11:11.822 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 273 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.823 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581592+0000", 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999999e-06 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581611+0000", 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8736000000000002e-05 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.824 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581652+0000", 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1421999999999997e-05 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681775+0000", 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100122909 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681951+0000", 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000175358 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:11.825 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681971+0000", 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0795e-05 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.858775+0000", 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17680337099999999 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.858953+0000", 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017833000000000001 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.826 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.363063+0000", 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50410986700000004 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:274 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.1137711350000004, 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.68252750799999995, 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.827 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 274 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.828 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681837+0000", 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.756e-06 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681852+0000", 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4566000000000001e-05 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681881+0000", 2024-04-18T23:11:11.829 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8651999999999999e-05 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.781982+0000", 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100101015 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782144+0000", 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162347 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:11.830 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782164+0000", 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9888999999999999e-05 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.959687+0000", 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17752280100000001 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.959899+0000", 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021188899999999999 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364362+0000", 2024-04-18T23:11:11.831 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40446359300000001 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:275 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0135867210000002, 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.78490300599999996, 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.832 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 275 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.833 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782021+0000", 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.497e-06 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782031+0000", 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8730000000000002e-06 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782049+0000", 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8016000000000002e-05 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.834 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782060+0000", 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1066e-05 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782264+0000", 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020450899999999999 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782285+0000", 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1066000000000001e-05 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.835 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364492+0000", 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58220711199999997 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364799+0000", 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030637600000000002 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.566922+0000", 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20212349099999999 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.836 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.0s0 42/31 ECSubReadReply(tid=60, attrs_read=0))", 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.859981+0000", 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9356247050000004, 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30148221800000002, 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.859981+0000", 2024-04-18T23:11:11.837 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.859981+0000", 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.859983+0000", 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.46e-06 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.838 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.860144+0000", 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016106000000000001 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.860171+0000", 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7665999999999999e-05 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960255+0000", 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100083952 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.839 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.161410+0000", 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20115445200000001 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.161463+0000", 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3628000000000002e-05 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.840 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.0s0 42/31 ECSubRead(tid=62, to_read={3:01c2b7c9:::smithi14230322-29:head=0,118784,0}, subchunks={3:01c2b7c9:::smithi14230322-29:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.960256+0000", 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8353493869999999, 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30172887199999998, 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960256+0000", 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.841 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960256+0000", 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960256+0000", 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960256+0000", 2024-04-18T23:11:11.842 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960320+0000", 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3544e-05 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.960350+0000", 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9543000000000002e-05 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.261714+0000", 2024-04-18T23:11:11.843 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30136491199999998 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.261985+0000", 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00027087299999999998 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.0s0 42/31 ECSubReadReply(tid=62, attrs_read=0))", 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.961274+0000", 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.8343316850000004, 2024-04-18T23:11:11.844 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40098276900000002, 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.961274+0000", 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.961274+0000", 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.845 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.961276+0000", 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.961523+0000", 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024748000000000001 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.961556+0000", 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2475000000000003e-05 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.846 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.961584+0000", 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8048999999999999e-05 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.362189+0000", 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40060523100000001 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.362257+0000", 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8053999999999996e-05 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.847 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.0s0 42/31 ECSubReadReply(tid=60, attrs_read=0))", 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.060867+0000", 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7347389060000005, 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30222712200000001, 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.060867+0000", 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.848 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.060867+0000", 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.060867+0000", 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.060867+0000", 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.849 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.060956+0000", 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8609999999999994e-05 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.060972+0000", 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6456e-05 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.362294+0000", 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30132205899999998 2024-04-18T23:11:11.850 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.363094+0000", 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00079999699999999999 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.0s0 42/31 ECSubReadReply(tid=62, attrs_read=0))", 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.062484+0000", 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7331215029999996, 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30079784599999998, 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.851 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.062484+0000", 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.062484+0000", 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.852 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.062486+0000", 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.567e-06 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.062695+0000", 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000208829 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.062723+0000", 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8155999999999999e-05 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.853 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.062736+0000", 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3485999999999999e-05 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.363176+0000", 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30043947199999999 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.363282+0000", 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010633600000000001 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.0s0 42/31 ECSubReadReply(tid=62, attrs_read=0))", 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.063120+0000", 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7324855159999997, 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30028416699999999, 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.063120+0000", 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.855 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.063120+0000", 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.063121+0000", 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7000000000000004e-07 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.063267+0000", 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000146291 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.856 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.063283+0000", 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6073e-05 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.163412+0000", 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10012895600000001 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.363344+0000", 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19993139300000001 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.857 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.363405+0000", 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0783999999999999e-05 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 42/31 ECSubWriteReply(tid=65, last_complete=42'12, committed=1, applied=1))", 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.369396+0000", 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4262102089999997, 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29802192700000002, 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.858 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369396+0000", 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369396+0000", 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369397+0000", 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6139999999999999e-06 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.859 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369406+0000", 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6959999999999994e-06 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369428+0000", 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1546e-05 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369441+0000", 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3409e-05 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.860 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.667352+0000", 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29791071499999999 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.667418+0000", 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5946999999999997e-05 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 42/31 ECSubWriteReply(tid=65, last_complete=42'12, committed=1, applied=1))", 2024-04-18T23:11:11.861 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.369521+0000", 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4260852399999999, 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29799410500000001, 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369521+0000", 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.862 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369521+0000", 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369522+0000", 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7300000000000004e-07 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369527+0000", 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1499999999999998e-06 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.863 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369544+0000", 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6803000000000002e-05 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.369555+0000", 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1516e-05 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.667490+0000", 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.297934747 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.864 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.667515+0000", 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4916e-05 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 42/31 ECSubWriteReply(tid=65, last_complete=42'12, committed=1, applied=1))", 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.370069+0000", 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.4255370239999996, 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29963008499999999, 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.865 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.370069+0000", 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.370069+0000", 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.370070+0000", 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.277e-06 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.866 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.370077+0000", 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9430000000000003e-06 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.370094+0000", 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7292999999999999e-05 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.370104+0000", 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6260000000000007e-06 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.867 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.668051+0000", 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29794710699999999 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669699+0000", 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016478389999999999 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.868 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:282 3.9s0 3:9e926f5e:::smithi14230322-42:4 [read 0~704512,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.589885+0000", 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.2057208749999999, 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30667666900000001, 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 282 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.869 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.589885+0000", 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.589885+0000", 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.589889+0000", 2024-04-18T23:11:11.870 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9299999999999996e-06 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.589908+0000", 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8746e-05 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.589952+0000", 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4765000000000003e-05 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.871 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.589981+0000", 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8464999999999999e-05 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590136+0000", 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000155055 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590148+0000", 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2489e-05 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.872 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.791007+0000", 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200858812 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.791266+0000", 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025883699999999999 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.896562+0000", 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10529557 2024-04-18T23:11:11.873 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:283 3.9s0 3:9e926f5e:::smithi14230322-42:4 [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.590022+0000", 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.2055838129999996, 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.307856146, 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.874 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 283 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590022+0000", 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590022+0000", 2024-04-18T23:11:11.875 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590023+0000", 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.11e-06 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590035+0000", 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2079000000000001e-05 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.876 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590048+0000", 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.325e-05 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590058+0000", 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6930000000000002e-06 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590244+0000", 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018551599999999999 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.877 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590253+0000", 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.9330000000000007e-06 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.791938+0000", 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201684948 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792112+0000", 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000174024 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.878 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.897878+0000", 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10576659300000001 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:284 3.9s0 3:9e926f5e:::smithi14230322-42:4 [read 0~704512,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.590073+0000", 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.2055330780000002, 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40667398300000002, 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:11.879 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 284 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590073+0000", 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.880 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590073+0000", 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590074+0000", 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.06e-07 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590078+0000", 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3970000000000004e-06 2024-04-18T23:11:11.881 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590088+0000", 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9820000000000003e-06 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.590100+0000", 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2234e-05 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.882 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.593066+0000", 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002966181 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.593086+0000", 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9729999999999999e-05 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792663+0000", 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199577434 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792933+0000", 2024-04-18T23:11:11.883 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00026999000000000001 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.996747+0000", 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20381332899999999 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:11.884 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:11.886 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:11:11.886 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:11.882+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:11.987 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:11.988+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:12.016 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:12.016 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:12.016 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:12.016 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:12.030 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:11:12.088 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:12.086+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:12.148 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:12.148 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:12.148 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:12.148 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:12.148 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:12.162 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:11:12.189 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:12.190+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:12.284 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:12.284 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:12.284 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:12.284 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:12.284 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.284 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s3 34/31 ECSubWrite(tid=0, reqid=client.4568.0:7, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.527180+0000", 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.755622254, 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0053240450000000003, 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527180+0000", 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.285 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527180+0000", 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527181+0000", 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5400000000000002e-07 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527299+0000", 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000118287 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.286 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527328+0000", 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9254000000000001e-05 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527351+0000", 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3119e-05 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527480+0000", 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012815299999999999 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.287 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527498+0000", 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8261e-05 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532392+0000", 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048938000000000002 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532504+0000", 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112817 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.288 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:8, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.529226+0000", 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.753576126, 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033638029999999998, 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.529226+0000", 2024-04-18T23:11:12.289 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.529226+0000", 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.529227+0000", 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2999999999999999e-07 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.529265+0000", 2024-04-18T23:11:12.290 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7972000000000002e-05 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.529290+0000", 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5191e-05 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.529300+0000", 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9469999999999998e-06 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.529469+0000", 2024-04-18T23:11:12.291 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016902700000000001 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.529494+0000", 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.427e-05 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532513+0000", 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030193960000000001 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532590+0000", 2024-04-18T23:11:12.292 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7169999999999995e-05 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 34/31 ECSubWrite(tid=1, reqid=client.4568.0:29, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.624154+0000", 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.658648175, 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0099604889999999995, 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.293 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624154+0000", 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624154+0000", 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624155+0000", 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6700000000000003e-07 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.294 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624183+0000", 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7778999999999999e-05 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624208+0000", 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5040000000000001e-05 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624245+0000", 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7076999999999998e-05 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.295 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.632990+0000", 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0087455689999999999 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.633000+0000", 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8810000000000008e-06 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634084+0000", 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010833990000000001 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.296 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634115+0000", 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1177000000000003e-05 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s3 34/31 ECSubWrite(tid=4, reqid=client.4568.0:8, at_version=0'0, trim_to=0'0, roll_forward_to=34'2))", 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.735474+0000", 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.547328836, 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021053310000000002, 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.297 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.735474+0000", 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.735474+0000", 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.735475+0000", 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.012e-06 2024-04-18T23:11:12.298 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.735486+0000", 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0875e-05 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.735499+0000", 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3664e-05 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.735516+0000", 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6305000000000001e-05 2024-04-18T23:11:12.299 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.735575+0000", 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9512999999999998e-05 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.735582+0000", 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4749999999999998e-06 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.737545+0000", 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019636390000000001 2024-04-18T23:11:12.300 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.737579+0000", 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3847999999999998e-05 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(osd.5.21:2 3.as0 3:53611365:::smithi14230322-34:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.645176+0000", 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.637626601000001, 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20476035200000001, 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.301 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645176+0000", 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645176+0000", 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645179+0000", 2024-04-18T23:11:12.302 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.92e-06 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645190+0000", 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0947e-05 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645212+0000", 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2075000000000001e-05 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645227+0000", 2024-04-18T23:11:12.303 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4989e-05 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645346+0000", 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011948800000000001 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645498+0000", 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015173899999999999 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.849936+0000", 2024-04-18T23:11:12.304 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20443819399999999 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.as0 36/31 ECSubRead(tid=8, to_read={3:53611365:::smithi14230322-34:head=0,139264,0}, subchunks={3:53611365:::smithi14230322-34:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.646120+0000", 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.636682233, 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027680579999999999, 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.305 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646120+0000", 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646120+0000", 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646120+0000", 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.306 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646120+0000", 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646220+0000", 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9219e-05 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646247+0000", 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7821999999999999e-05 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.307 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647977+0000", 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0017294770000000001 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648888+0000", 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00091153999999999996 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/31 ECSubReadReply(tid=8, attrs_read=0))", 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647184+0000", 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.635618644000001, 2024-04-18T23:11:12.308 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001877121, 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647184+0000", 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647184+0000", 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.309 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647185+0000", 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5099999999999999e-06 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647678+0000", 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049267400000000002 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647711+0000", 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2381e-05 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.310 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647729+0000", 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8963000000000001e-05 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648971+0000", 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012414570000000001 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649061+0000", 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0136000000000006e-05 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.311 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/31 ECSubReadReply(tid=8, attrs_read=0))", 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647408+0000", 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.635394641, 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10186719900000001, 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647408+0000", 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.312 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647408+0000", 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647409+0000", 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2440000000000001e-06 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648186+0000", 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00077665600000000003 2024-04-18T23:11:12.313 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648203+0000", 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6656000000000001e-05 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648218+0000", 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5085999999999999e-05 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749203+0000", 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100984977 2024-04-18T23:11:12.314 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749275+0000", 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.258e-05 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/31 ECSubReadReply(tid=8, attrs_read=0))", 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.648647+0000", 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.63415548, 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100859843, 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.315 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648647+0000", 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648647+0000", 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.316 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648648+0000", 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648917+0000", 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026861399999999997 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648939+0000", 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1769000000000001e-05 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.317 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648956+0000", 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7476e-05 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749394+0000", 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100437887 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749507+0000", 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112717 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.318 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/31 ECSubReadReply(tid=8, attrs_read=0))", 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.633939743999999, 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20110777299999999, 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.319 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.320 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648921+0000", 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8091000000000003e-05 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648981+0000", 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0117000000000001e-05 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.321 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.849495+0000", 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.200514265 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.849971+0000", 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00047530000000000001 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 36/31 ECSubWrite(tid=25, reqid=client.4568.0:185, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.648984+0000", 2024-04-18T23:11:12.322 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.633818243, 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019780029999999999, 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648984+0000", 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648984+0000", 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.323 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648985+0000", 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1170000000000001e-06 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649043+0000", 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7553999999999998e-05 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649065+0000", 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1699e-05 2024-04-18T23:11:12.324 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649083+0000", 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7869000000000001e-05 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649209+0000", 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012653799999999999 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649223+0000", 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3565e-05 2024-04-18T23:11:12.325 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.650896+0000", 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001673477 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.650962+0000", 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6184000000000002e-05 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.326 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.9s3 36/31 ECSubRead(tid=37, to_read={3:906aad0a:::smithi14230322-27:head=0,110592,0}, subchunks={3:906aad0a:::smithi14230322-27:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.649071+0000", 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.633731447000001, 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101279314, 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649071+0000", 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.327 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649071+0000", 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649072+0000", 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0699999999999996e-07 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649089+0000", 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7071999999999998e-05 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.328 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649128+0000", 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9168999999999997e-05 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749239+0000", 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100111065 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749398+0000", 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015834000000000001 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.329 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.750350+0000", 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000952961 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 36/31 ECSubWrite(tid=27, reqid=client.4568.0:187, at_version=36'15, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.751178+0000", 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.531624653, 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10144216, 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.330 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751178+0000", 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751178+0000", 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751179+0000", 2024-04-18T23:11:12.331 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.333e-06 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751215+0000", 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5611999999999997e-05 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751244+0000", 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9422000000000001e-05 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751267+0000", 2024-04-18T23:11:12.332 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2779e-05 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.851479+0000", 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10021179400000001 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.851497+0000", 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8128999999999999e-05 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.852539+0000", 2024-04-18T23:11:12.333 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010423380000000001 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.852620+0000", 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0753000000000004e-05 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 36/31 ECSubWrite(tid=29, reqid=client.4568.0:202, at_version=36'17, trim_to=0'0, roll_forward_to=36'15))", 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:01.171077+0000", 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.111725946, 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028450530000000002, 2024-04-18T23:11:12.334 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.171077+0000", 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.171077+0000", 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.335 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.171079+0000", 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9800000000000001e-06 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.171205+0000", 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012677399999999999 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.171271+0000", 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5519999999999996e-05 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.336 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.171342+0000", 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0980000000000001e-05 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.171553+0000", 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000211494 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.171575+0000", 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1761999999999999e-05 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.337 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.173848+0000", 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022723629999999999 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.173922+0000", 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4179999999999998e-05 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s2 42/31 ECSubWrite(tid=181, reqid=osd.13.0:181, at_version=42'16, trim_to=0'0, roll_forward_to=40'14))", 2024-04-18T23:11:12.338 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.589402+0000", 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.6934008470000004, 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021630669999999999, 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.589402+0000", 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.339 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.589402+0000", 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.589402+0000", 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0999999999999999e-07 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.589440+0000", 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8232000000000003e-05 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.340 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.589466+0000", 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5718999999999999e-05 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.589483+0000", 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6736e-05 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.589649+0000", 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016622300000000001 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.341 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.589675+0000", 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6041999999999999e-05 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.591497+0000", 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018219919999999999 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.591565+0000", 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7712999999999994e-05 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.342 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s3 42/31 ECSubWrite(tid=61, reqid=osd.14.0:61, at_version=42'25, trim_to=0'0, roll_forward_to=40'23))", 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.959544+0000", 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.3232585620000004, 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018342300000000001, 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959544+0000", 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.343 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959544+0000", 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959546+0000", 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.306e-06 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959603+0000", 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7065000000000003e-05 2024-04-18T23:11:12.344 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959646+0000", 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2284999999999999e-05 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959671+0000", 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4998999999999999e-05 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959794+0000", 2024-04-18T23:11:12.345 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000123521 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959814+0000", 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9303000000000001e-05 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.961301+0000", 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001487824 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.961378+0000", 2024-04-18T23:11:12.346 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6927000000000004e-05 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s3 42/31 ECSubWrite(tid=63, reqid=osd.14.0:61, at_version=0'0, trim_to=0'0, roll_forward_to=42'25))", 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:07.263035+0000", 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.0197680939999998, 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10154968, 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.347 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263035+0000", 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263035+0000", 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263037+0000", 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1069999999999999e-06 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.348 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263061+0000", 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4131999999999999e-05 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263092+0000", 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1399000000000002e-05 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263109+0000", 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.694e-05 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.349 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.363418+0000", 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100308642 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.363434+0000", 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6124999999999999e-05 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.364491+0000", 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010569080000000001 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.350 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.364584+0000", 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3426999999999998e-05 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 46/31 ECSubWrite(tid=40, reqid=client.4568.0:323, at_version=46'23, trim_to=0'0, roll_forward_to=40'21))", 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:11.120534+0000", 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.1622683810000001, 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027153500000000001, 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.351 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120534+0000", 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120534+0000", 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120536+0000", 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9400000000000001e-06 2024-04-18T23:11:12.352 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120581+0000", 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5244999999999998e-05 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120614+0000", 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2659e-05 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120638+0000", 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3468999999999998e-05 2024-04-18T23:11:12.353 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120816+0000", 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017892 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120857+0000", 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0055000000000002e-05 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.123176+0000", 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002319658 2024-04-18T23:11:12.354 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.123250+0000", 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3404000000000004e-05 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 46/31 ECSubWrite(tid=41, reqid=client.4568.0:323, at_version=0'0, trim_to=0'0, roll_forward_to=46'23))", 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:11.424855+0000", 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.85794751199999997, 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.1019087, 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.355 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424855+0000", 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424855+0000", 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424857+0000", 2024-04-18T23:11:12.356 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2469999999999998e-06 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424910+0000", 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2784000000000001e-05 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424957+0000", 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7141999999999997e-05 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.425023+0000", 2024-04-18T23:11:12.357 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5814000000000004e-05 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525324+0000", 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100300527 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525339+0000", 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5523999999999999e-05 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.358 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.526710+0000", 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001370511 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.526764+0000", 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4150999999999997e-05 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 46/31 ECSubWrite(tid=42, reqid=client.4568.0:330, at_version=46'25, trim_to=0'0, roll_forward_to=46'23))", 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:11.431071+0000", 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.85173117099999995, 2024-04-18T23:11:12.359 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.096741165000000004, 2024-04-18T23:11:12.360 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.360 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.360 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.360 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.360 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.360 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.360 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431071+0000", 2024-04-18T23:11:12.361 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.361 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.361 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.361 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431071+0000", 2024-04-18T23:11:12.361 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.361 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.361 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431073+0000", 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5230000000000001e-06 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431138+0000", 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4714000000000004e-05 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431167+0000", 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9456000000000001e-05 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.362 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431190+0000", 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3013000000000001e-05 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525828+0000", 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.094638112999999996 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525842+0000", 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3963e-05 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.363 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.527743+0000", 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001900765 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.527813+0000", 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9617999999999997e-05 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.364 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:12.366 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:11:12.367 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:12.364+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:11:12.466 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:12.464+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:12.537 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:12.537 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:12.537 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:12.537 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:12.552 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:11:12.594 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:12.592+0000 7fdb75f98640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 25341) UID: 0 2024-04-18T23:11:12.620 INFO:tasks.rados.rados.0.smithi142.stdout:130: done (2 left) 2024-04-18T23:11:12.620 INFO:tasks.rados.rados.0.smithi142.stdout:131: setattr oid 39 current snap is 7 2024-04-18T23:11:12.620 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 41 v 25 (ObjNum 40 snap 0 seq_num 40) dirty exists 2024-04-18T23:11:12.620 INFO:tasks.rados.rados.0.smithi142.stdout:126: finishing rollback tid 0 to smithi14230322-49 2024-04-18T23:11:12.621 INFO:tasks.rados.rados.0.smithi142.stdout:126: finishing rollback tid 1 to smithi14230322-49 2024-04-18T23:11:12.621 INFO:tasks.rados.rados.0.smithi142.stdout:126: finishing rollback tid 2 to smithi14230322-49 2024-04-18T23:11:12.621 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 49 v 31 (ObjNum 48 snap 0 seq_num 48) dirty exists 2024-04-18T23:11:12.623 INFO:tasks.rados.rados.0.smithi142.stdout:126: done (2 left) 2024-04-18T23:11:12.623 INFO:tasks.rados.rados.0.smithi142.stdout:128: done (1 left) 2024-04-18T23:11:12.623 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 49 current snap is 7 2024-04-18T23:11:12.624 INFO:tasks.rados.rados.0.smithi142.stdout:132: seq_num 64 ranges {819200=704512,1523712=376832} 2024-04-18T23:11:12.624 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-04-18T23:11:12.634 INFO:tasks.rados.rados.0.smithi142.stdout:132: writing smithi14230322-49 from 819200 to 1523712 tid 1 2024-04-18T23:11:12.638 INFO:tasks.rados.rados.0.smithi142.stdout:132: writing smithi14230322-49 from 1523712 to 1900544 tid 2 2024-04-18T23:11:12.638 INFO:tasks.rados.rados.0.smithi142.stdout:133: read oid 48 snap 2 2024-04-18T23:11:12.638 INFO:tasks.rados.rados.0.smithi142.stdout:133: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:12.638 INFO:tasks.rados.rados.0.smithi142.stdout:134: rollback oid 27 current snap is 7 2024-04-18T23:11:12.638 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 27 to 2 2024-04-18T23:11:12.638 INFO:tasks.rados.rados.0.smithi142.stdout:135: delete oid 17 current snap is 7 2024-04-18T23:11:12.668 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:12.664+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:12.714 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:12.714 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:12.714 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:12.714 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:12.714 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:12.730 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:11:12.770 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:12.766+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:12.800 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:12.871 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:12.870+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:12.898 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:28 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~163840 in=163840b,stat out=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.523499+0000", 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.375105305000002, 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31231229100000002, 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.899 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 28 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523499+0000", 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523499+0000", 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523501+0000", 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.125e-06 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523671+0000", 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000170767 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523688+0000", 2024-04-18T23:11:12.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6565000000000001e-05 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523699+0000", 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0825999999999999e-05 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523855+0000", 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015598399999999999 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523862+0000", 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5429999999999996e-06 2024-04-18T23:11:12.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532410+0000", 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0085475919999999997 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532697+0000", 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028755700000000003 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.533837+0000", 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001139999 2024-04-18T23:11:12.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.536785+0000", 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029477489999999999 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.835735+0000", 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29894969300000002 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.835811+0000", 2024-04-18T23:11:12.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5890999999999996e-05 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:29 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [setxattr _header (56) in=63b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.523729+0000", 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.374874480999999, 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31339988000000002, 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.905 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 29 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523729+0000", 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523729+0000", 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523730+0000", 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5899999999999995e-07 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523734+0000", 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.211e-06 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523746+0000", 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1398e-05 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523771+0000", 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4817e-05 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523929+0000", 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015801500000000001 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523934+0000", 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3240000000000002e-06 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.534690+0000", 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010756533 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.534992+0000", 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030151700000000001 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.536263+0000", 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012710499999999999 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.540150+0000", 2024-04-18T23:11:12.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038867789999999999 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.837034+0000", 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29688388500000001 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.837129+0000", 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5592000000000002e-05 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:185 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.646795+0000", 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.251808935, 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40739746900000001, 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 185 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646795+0000", 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646795+0000", 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646797+0000", 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.95e-06 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646805+0000", 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7659999999999998e-06 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646830+0000", 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5228999999999998e-05 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646850+0000", 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0662e-05 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647189+0000", 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00033802099999999998 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647469+0000", 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028023000000000002 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648937+0000", 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014680369999999999 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751937+0000", 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10300072 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.054067+0000", 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30212936699999998 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.054192+0000", 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000125487 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.915 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:186 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.646882+0000", 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.251721449, 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40783060199999999, 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 186 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.916 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646882+0000", 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646882+0000", 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646884+0000", 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.513e-06 2024-04-18T23:11:12.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646894+0000", 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7229999999999999e-06 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646913+0000", 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9769e-05 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646929+0000", 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5586e-05 2024-04-18T23:11:12.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.746864+0000", 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099935144000000004 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.747144+0000", 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028019299999999998 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.748787+0000", 2024-04-18T23:11:12.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0016425890000000001 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752698+0000", 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0039113430000000003 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.054522+0000", 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30182420799999998 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.054713+0000", 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019053399999999999 2024-04-18T23:11:12.920 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:187 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647001+0000", 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.251602582, 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40801147199999999, 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 187 2024-04-18T23:11:12.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647001+0000", 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647001+0000", 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647002+0000", 2024-04-18T23:11:12.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2270000000000001e-06 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647009+0000", 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8229999999999997e-06 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647033+0000", 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3646e-05 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647052+0000", 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9279e-05 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749808+0000", 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102756006 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.750075+0000", 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026714700000000002 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751371+0000", 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001295631 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752960+0000", 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015885680000000001 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.054910+0000", 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30195069600000002 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.055013+0000", 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010244899999999999 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.925 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:202 3.ds0 3:ba4438ef:::smithi14230322-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.251528002000001, 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.726850518, 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.926 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 202 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647077+0000", 2024-04-18T23:11:12.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.497e-06 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647086+0000", 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5490000000000006e-06 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647105+0000", 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8933e-05 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647120+0000", 2024-04-18T23:11:12.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4942e-05 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752033+0000", 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10491311 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752334+0000", 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030070900000000002 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.072768+0000", 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32043411399999999 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.075547+0000", 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027797260000000002 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.373671+0000", 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29812367699999998 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.373926+0000", 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025526100000000001 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:203 3.ds0 3:ba4438ef:::smithi14230322-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647172+0000", 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.25143205, 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.42781979999999997, 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.931 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 203 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647172+0000", 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647172+0000", 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647174+0000", 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8369999999999999e-06 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647180+0000", 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9529999999999996e-06 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647201+0000", 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1501999999999998e-05 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647218+0000", 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.647e-05 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.753422+0000", 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10620457699999999 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.753567+0000", 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014483399999999999 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.934 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.074716+0000", 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.321149236 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.074851+0000", 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000135258 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.074992+0000", 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014013299999999999 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 36/31 ECSubWriteReply(tid=25, last_complete=36'13, committed=1, applied=1))", 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.752761+0000", 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.145843199, 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30066203499999999, 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752761+0000", 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752761+0000", 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752762+0000", 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8199999999999999e-06 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752771+0000", 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0129999999999999e-06 2024-04-18T23:11:12.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752796+0000", 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4528999999999998e-05 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.852890+0000", 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100094237 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.053373+0000", 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20048306199999999 2024-04-18T23:11:12.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.053423+0000", 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9373999999999999e-05 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.5s4 43/31 ECSubWrite(tid=33, reqid=osd.15.0:33, at_version=43'19, trim_to=0'0, roll_forward_to=42'17))", 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:07.680020+0000", 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.218583958, 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20458173499999999, 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.939 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.680020+0000", 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.680020+0000", 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.680020+0000", 2024-04-18T23:11:12.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5700000000000002e-07 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.680073+0000", 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2890999999999999e-05 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.680110+0000", 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6891999999999998e-05 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.680128+0000", 2024-04-18T23:11:12.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8022000000000001e-05 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.880494+0000", 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20036605599999999 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.880510+0000", 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5879000000000002e-05 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.884522+0000", 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040114039999999997 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.884602+0000", 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0034e-05 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:317 3.ds0 3:ba4438ef:::smithi14230322-48:7 [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609553+0000", 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.2890509730000002, 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30711334000000001, 2024-04-18T23:11:12.943 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 317 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609553+0000", 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609553+0000", 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609555+0000", 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5399999999999998e-06 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609584+0000", 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8962000000000002e-05 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609618+0000", 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3538999999999999e-05 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609643+0000", 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4933e-05 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609806+0000", 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000163194 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609819+0000", 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2523e-05 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.710648+0000", 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100829714 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.710952+0000", 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030364500000000001 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.916666+0000", 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20571428999999999 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:318 3.ds0 3:ba4438ef:::smithi14230322-48:7 [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609705+0000", 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.288898755, 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40842472699999999, 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:12.948 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 318 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609705+0000", 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609705+0000", 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609707+0000", 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7e-06 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609725+0000", 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7992999999999999e-05 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609740+0000", 2024-04-18T23:11:12.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5259000000000001e-05 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609757+0000", 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6806000000000001e-05 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609990+0000", 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000232721 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.610013+0000", 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3203000000000001e-05 2024-04-18T23:11:12.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.712982+0000", 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102969296 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.713166+0000", 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000184217 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.018130+0000", 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30496353199999998 2024-04-18T23:11:12.952 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:319 3.ds0 3:ba4438ef:::smithi14230322-48:7 [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609786+0000", 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.2888175479999999, 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50925149999999997, 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.953 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 319 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609786+0000", 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609786+0000", 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.954 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609788+0000", 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3999999999999999e-06 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609795+0000", 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3200000000000002e-06 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609813+0000", 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8493000000000001e-05 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609830+0000", 2024-04-18T23:11:12.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6062e-05 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.610105+0000", 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000275467 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.610126+0000", 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1262000000000001e-05 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.814838+0000", 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.204711587 2024-04-18T23:11:12.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.815034+0000", 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000196579 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119038+0000", 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30400333000000002 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.957 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:323 3.ds0 3:b8bdcfdb:::smithi14230322-44:head [delete] snapc 8=[8,7,4] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.2887428970000001, 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81471389100000002, 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 323 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.958 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609863+0000", 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6360000000000001e-06 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609869+0000", 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4699999999999999e-06 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609890+0000", 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0496000000000002e-05 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609905+0000", 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5539000000000001e-05 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.614202+0000", 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042970489999999998 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.614219+0000", 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7373000000000001e-05 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119137+0000", 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50491712899999996 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119293+0000", 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015688099999999999 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120709+0000", 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014158669999999999 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.122707+0000", 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019980140000000002 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.962 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424382+0000", 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30167486399999999 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424575+0000", 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000192573 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.ds0 46/31 ECSubReadReply(tid=37, attrs_read=0))", 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.714256+0000", 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.1843482540000001, 2024-04-18T23:11:12.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20243204400000001, 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.714256+0000", 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.714256+0000", 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.714256+0000", 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.714256+0000", 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.813785+0000", 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099529475000000006 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.813795+0000", 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0214000000000001e-05 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.916300+0000", 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102504846 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.916688+0000", 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00038750900000000002 2024-04-18T23:11:12.966 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.ds0 46/31 ECSubReadReply(tid=38, attrs_read=0))", 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.817049+0000", 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.0815544519999998, 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20110161700000001, 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.817049+0000", 2024-04-18T23:11:12.967 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.817049+0000", 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.817049+0000", 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.817049+0000", 2024-04-18T23:11:12.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.817115+0000", 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5758000000000003e-05 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.817139+0000", 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3799e-05 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.017799+0000", 2024-04-18T23:11:12.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.200660229 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.018151+0000", 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00035183099999999999 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.ds0 46/31 ECSubReadReply(tid=39, attrs_read=0))", 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.916867+0000", 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.981736444, 2024-04-18T23:11:12.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.202196082, 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.916867+0000", 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.916867+0000", 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.916867+0000", 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.916867+0000", 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.916948+0000", 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0734000000000003e-05 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.916962+0000", 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4367e-05 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.118700+0000", 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.201737204 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119063+0000", 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00036377700000000002 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.973 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 46/31 ECSubWriteReply(tid=40, last_complete=46'23, committed=1, applied=1))", 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:11.123667+0000", 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.7749366530000001, 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30181451799999998, 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.123667+0000", 2024-04-18T23:11:12.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.123667+0000", 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.123668+0000", 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.153e-06 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.975 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.123676+0000", 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4229999999999999e-06 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.123699+0000", 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3125999999999998e-05 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.977 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.123713+0000", 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4396000000000001e-05 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424254+0000", 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30054069700000002 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.425482+0000", 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001227723 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.978 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:330 3.ds0 3:bb40a4d8:::smithi14230322-41:head [create,setxattr pUpejN\\XeR[ScpqbhnaXajbqbVcs]\\YUaoVqmdSQ (16807) in=40b,setxattr ^TXcXY\\i\\UaegqjXsVZRnYhXQcQtNan^ZkmbhgfY (349) in=40b,setxattr ekSnl]Q^sZQXVqUlaf`k_aifQmosa`q\\TjqRt_Qf (1227) in=40b,setxattr \\iWlX_lYknVg`YeraZ]fTRtp[SmcmU\\fi`W]WrbQ (6244) in=40b,setxattr jXafahNhrkaS]`RTUfZsViWZQb_fe[hNa`eaSXaP (104) in=40b,setxattr SQfdcs\\p[W^UXTPlNX^ZpfRc`f[oTemUqrW]noj\\ (1227) in=40b,setxattr \\^nWTlXbgpUbdWkgbnaQn`^hmQ[]WmV]lU[SaXU] (1542) in=40b,setxattr dVRspsbTY\\[_ajteqtXtW[`oNOrUtVnqhPan_ePQ (314) in=40b,setxattr tQcXlPoWcYqTci^srZesYV[VgVaRshSrZtc\\mZa^ (67) in=40b,setxattr s]QeT_cN\\hZPf_Whl\\VZhhldY`gN^hNQ[rbdZ\\aZ (244) in=40b,setxattr Sga\\QYQOYspmkSbrWjWZXUeqopRefVUPXn]SjYcY (1719) in=40b,setxattr [NiqOQcWdfT`oVaqXYf_dm`h_g]OscjStsShYaUN (876) in=40b,setxattr Ok_\\_emNX_rOSq`pl[ijceSO_WNtXVhjhtTPolkN (1647) in=40b,setxattr saRsYUrZhdhmY^k^i[Rph][cqVtPsi`WarjOnYO^ (1155) in=40b,setxattr cY\\acssfT[QaU\\pigijbXs\\ROcY__s[R_geknqaa (1472) in=40b,setxattr WfSmRUdrW[ojWRr_kTre\\ns`t[cOrTVTNankd[OP (350) in=40b,setxattr T`ce]ZdmspasZU]X[c[socq[elaihoTSQi_WieWt (1226) in=40b,setxattr piRXhWpPba^f_P_Z[\\`\\Ugj[kRno\\VgbWQlndVa] (1718) in=40b,setxattr lV]d\\p[Pbb_ShS]kXZmPXqpq`QhRa`cXeNpTNciq (875) in=40b,setxattr cdtSkTmQRaR]lfSnj]qi[pkO`[NRbaqUWsnj]XiO (314) in=40b] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:11.428020+0000", 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.4705835949999999, 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20741780400000001, 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 330 2024-04-18T23:11:12.979 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.428020+0000", 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.428020+0000", 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.428023+0000", 2024-04-18T23:11:12.980 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4889999999999998e-06 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.428097+0000", 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4331000000000004e-05 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.428126+0000", 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8469000000000001e-05 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.428144+0000", 2024-04-18T23:11:12.981 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8649000000000001e-05 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.428304+0000", 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000160285 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.428649+0000", 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00034505400000000002 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:12.982 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431503+0000", 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028539110000000002 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.434057+0000", 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025540419999999999 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.635280+0000", 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20122271999999999 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.635438+0000", 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000157854 2024-04-18T23:11:12.983 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:338 3.ds0 3:ba4438ef:::smithi14230322-48:4 [read 0~32768,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:12.640147+0000", 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.25845702999999998, 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.206247717, 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 338 2024-04-18T23:11:12.984 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640147+0000", 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640147+0000", 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640150+0000", 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2030000000000001e-06 2024-04-18T23:11:12.985 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640168+0000", 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7600000000000001e-05 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640204+0000", 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5970999999999998e-05 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640221+0000", 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7937000000000001e-05 2024-04-18T23:11:12.986 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.740432+0000", 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10021060599999999 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.740447+0000", 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4623999999999999e-05 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.741967+0000", 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001520167 2024-04-18T23:11:12.987 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.742148+0000", 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018129999999999999 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.846395+0000", 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104246309 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.988 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:339 3.ds0 3:ba4438ef:::smithi14230322-48:4 [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:12.640266+0000", 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.25833748499999998, 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.206314419, 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 339 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:12.989 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640266+0000", 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640266+0000", 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640267+0000", 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.114e-06 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640273+0000", 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4029999999999998e-06 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640285+0000", 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.198e-05 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.640293+0000", 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2619999999999998e-06 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.740611+0000", 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100318251 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.740632+0000", 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0622e-05 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.742738+0000", 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021062939999999999 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:12.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.742898+0000", 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015966099999999999 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.846581+0000", 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103682832 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:12.993 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:12.994 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:12.995 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:12.995 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:11:12.996 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 39 v 16 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-18T23:11:12.996 INFO:tasks.rados.rados.0.smithi142.stdout:131: done (4 left) 2024-04-18T23:11:12.996 INFO:tasks.rados.rados.0.smithi142.stdout:135: done (3 left) 2024-04-18T23:11:12.996 INFO:tasks.rados.rados.0.smithi142.stdout:136: delete oid 19 current snap is 7 2024-04-18T23:11:12.996 INFO:tasks.rados.rados.0.smithi142.stdout:133: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:12.997 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:12.996+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:13.096 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:13.092+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:13.123 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:13.123 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:13.123 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:13.136 INFO:tasks.rados.rados.0.smithi142.stdout:132: finishing write tid 1 to smithi14230322-49 2024-04-18T23:11:13.136 INFO:tasks.rados.rados.0.smithi142.stdout:132: finishing write tid 2 to smithi14230322-49 2024-04-18T23:11:13.137 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:11:13.145 INFO:tasks.rados.rados.0.smithi142.stdout:133: done (3 left) 2024-04-18T23:11:13.145 INFO:tasks.rados.rados.0.smithi142.stdout:136: done (2 left) 2024-04-18T23:11:13.145 INFO:tasks.rados.rados.0.smithi142.stdout:137: copy_from oid 12 from oid 8 current snap is 7 2024-04-18T23:11:13.145 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 38 current snap is 7 2024-04-18T23:11:13.146 INFO:tasks.rados.rados.0.smithi142.stdout:138: seq_num 67 ranges {81920=753664,835584=376832} 2024-04-18T23:11:13.159 INFO:tasks.rados.rados.0.smithi142.stdout:138: writing smithi14230322-38 from 81920 to 835584 tid 1 2024-04-18T23:11:13.160 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:13.160 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:13.165 INFO:tasks.rados.rados.0.smithi142.stdout:138: writing smithi14230322-38 from 835584 to 1212416 tid 2 2024-04-18T23:11:13.166 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 35 current snap is 7 2024-04-18T23:11:13.166 INFO:tasks.rados.rados.0.smithi142.stdout:139: seq_num 68 ranges {573440=770048,1343488=376832} 2024-04-18T23:11:13.187 INFO:tasks.rados.rados.0.smithi142.stdout:139: writing smithi14230322-35 from 573440 to 1343488 tid 1 2024-04-18T23:11:13.190 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:13.190 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:13.194 INFO:tasks.rados.rados.0.smithi142.stdout:139: writing smithi14230322-35 from 1343488 to 1720320 tid 2 2024-04-18T23:11:13.194 INFO:tasks.rados.rados.0.smithi142.stdout:140: copy_from oid 31 from oid 19 current snap is 7 2024-04-18T23:11:13.194 INFO:tasks.rados.rados.0.smithi142.stdout:141: copy_from oid 20 from oid 2 current snap is 7 2024-04-18T23:11:13.194 INFO:tasks.rados.rados.0.smithi142.stdout:142: snap_create 2024-04-18T23:11:13.196 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:13.196+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:13.221 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:13.268 INFO:tasks.rados.rados.0.smithi142.stdout:132: finishing write tid 3 to smithi14230322-49 2024-04-18T23:11:13.275 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:13.275 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:13.275 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:13.275 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:13.275 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:13.289 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:11:13.297 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:13.294+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:13.298 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:13.298 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:13.298 INFO:tasks.rados.rados.0.smithi142.stdout:137: finishing copy_from racing read to smithi14230322-12 2024-04-18T23:11:13.328 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:13.328 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:13.331 INFO:tasks.rados.rados.0.smithi142.stdout:134: finishing rollback tid 0 to smithi14230322-27 2024-04-18T23:11:13.359 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:13.360 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:13.389 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:13.390 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:13.398 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:13.396+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.1s1 33/31 ECSubWrite(tid=0, reqid=client.4568.0:1, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.523912+0000", 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.893976231, 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19916341000000001, 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.419 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523912+0000", 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523912+0000", 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523913+0000", 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.48e-06 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.523979+0000", 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5753000000000006e-05 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.524006+0000", 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6936000000000001e-05 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.524030+0000", 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4227999999999999e-05 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527886+0000", 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038554499999999998 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527907+0000", 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1202000000000001e-05 2024-04-18T23:11:13.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723019+0000", 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.195111907 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.723075+0000", 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6453999999999999e-05 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.1s1 33/31 ECSubWrite(tid=3, reqid=client.4568.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.527595+0000", 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.890292680999998, 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29580373399999998, 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527595+0000", 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527595+0000", 2024-04-18T23:11:13.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527596+0000", 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1600000000000001e-07 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527623+0000", 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6698000000000001e-05 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527644+0000", 2024-04-18T23:11:13.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1946e-05 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527665+0000", 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0859000000000001e-05 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.722634+0000", 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19496872100000001 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:13.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.722649+0000", 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4832e-05 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.823340+0000", 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100690934 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.823399+0000", 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9228000000000003e-05 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:191 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560221+0000", 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.857667372, 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10621468100000001, 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 191 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560221+0000", 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560221+0000", 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560244+0000", 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3051999999999999e-05 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560269+0000", 2024-04-18T23:11:13.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5326000000000001e-05 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560345+0000", 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5486000000000002e-05 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560371+0000", 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6758999999999999e-05 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560573+0000", 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000201547 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560587+0000", 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3943e-05 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.661427+0000", 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100840518 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.661594+0000", 2024-04-18T23:11:13.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016658499999999999 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.666435+0000", 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048414649999999997 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:192 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.857450212, 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30501581700000002, 2024-04-18T23:11:13.432 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 192 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560439+0000", 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6029999999999999e-06 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560450+0000", 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0835999999999999e-05 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560478+0000", 2024-04-18T23:11:13.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7619000000000001e-05 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560489+0000", 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0998999999999999e-05 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560680+0000", 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019082800000000001 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560688+0000", 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2260000000000005e-06 2024-04-18T23:11:13.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662105+0000", 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10141672 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662220+0000", 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115872 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.865454+0000", 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20323311399999999 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.436 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:193 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.857341466999999, 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30581751899999998, 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 193 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560548+0000", 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560554+0000", 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2230000000000004e-06 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560566+0000", 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2622999999999999e-05 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560575+0000", 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1589999999999993e-06 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.439 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563654+0000", 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030794709999999999 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563665+0000", 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0708e-05 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662682+0000", 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099017665000000005 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.663181+0000", 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049841999999999996 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.866364+0000", 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20318328699999999 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:209 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560597+0000", 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.857290787, 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.206586359, 2024-04-18T23:11:13.441 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 209 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560597+0000", 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560597+0000", 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560598+0000", 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6000000000000003e-07 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560603+0000", 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.057e-06 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560617+0000", 2024-04-18T23:11:13.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3801999999999999e-05 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560625+0000", 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6869999999999994e-06 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560749+0000", 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012451399999999999 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560769+0000", 2024-04-18T23:11:13.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0222000000000001e-05 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.564039+0000", 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003269831 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.564169+0000", 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000130147 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767184+0000", 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20301433899999999 2024-04-18T23:11:13.445 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:210 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.857243859, 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.41278097800000002, 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:13.446 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 210 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560645+0000", 2024-04-18T23:11:13.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1829999999999999e-06 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560652+0000", 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3500000000000002e-06 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560669+0000", 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6833e-05 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560682+0000", 2024-04-18T23:11:13.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3585e-05 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560856+0000", 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017400200000000001 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560874+0000", 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8386e-05 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767285+0000", 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20641079700000001 2024-04-18T23:11:13.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767423+0000", 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013740399999999999 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.973425+0000", 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20600243800000001 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:211 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.857155562999999, 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51049911299999995, 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 211 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.451 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560734+0000", 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-06 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560741+0000", 2024-04-18T23:11:13.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1559999999999998e-06 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560758+0000", 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7382999999999999e-05 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560775+0000", 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7051999999999999e-05 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560940+0000", 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164659 2024-04-18T23:11:13.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560949+0000", 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7630000000000006e-06 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767943+0000", 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.206994276 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.768107+0000", 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164358 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.071232+0000", 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30312427600000003 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/31 ECSubWrite(tid=54, reqid=client.4568.0:188, at_version=36'19, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.657889+0000", 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.759998962999999, 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.110021097, 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.455 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.657889+0000", 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.657889+0000", 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.657890+0000", 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.13e-07 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.658029+0000", 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000139624 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.658069+0000", 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9415000000000001e-05 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.658097+0000", 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8432000000000002e-05 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.764267+0000", 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.106170113 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.764281+0000", 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4002000000000001e-05 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767822+0000", 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0035405520000000002 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767910+0000", 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8446e-05 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.cs0 36/31 ECSubReadReply(tid=14, attrs_read=0))", 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.664438+0000", 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.753449594999999, 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20104434600000001, 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.664438+0000", 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.664438+0000", 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.664438+0000", 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.664438+0000", 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.664483+0000", 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4567000000000002e-05 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.764575+0000", 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10009251700000001 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.864908+0000", 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100332722 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.865483+0000", 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00057454000000000001 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.cs0 36/31 ECSubReadReply(tid=15, attrs_read=0))", 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.665121+0000", 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.752767172, 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20128283899999999, 2024-04-18T23:11:13.462 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.665121+0000", 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.665121+0000", 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.665121+0000", 2024-04-18T23:11:13.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.665121+0000", 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.764637+0000", 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099516411999999999 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.764653+0000", 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5919000000000001e-05 2024-04-18T23:11:13.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.865579+0000", 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10092602000000001 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.866404+0000", 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00082448799999999996 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.cs0 36/31 ECSubReadReply(tid=15, attrs_read=0))", 2024-04-18T23:11:13.465 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.724424+0000", 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.693463573000001, 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.140441023, 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.724424+0000", 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.724424+0000", 2024-04-18T23:11:13.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.724426+0000", 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.84e-06 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.724489+0000", 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2823999999999993e-05 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.724517+0000", 2024-04-18T23:11:13.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7909e-05 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.724530+0000", 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3326000000000001e-05 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.864793+0000", 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.14026286399999999 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.864865+0000", 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2260000000000003e-05 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.468 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/31 ECSubWrite(tid=58, reqid=client.4568.0:196, at_version=36'21, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.761778+0000", 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.656110399999999, 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20846394800000001, 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761778+0000", 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761778+0000", 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761779+0000", 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.646e-06 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761841+0000", 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1684000000000001e-05 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761871+0000", 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0139000000000001e-05 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761888+0000", 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6439e-05 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.765276+0000", 2024-04-18T23:11:13.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033887309999999999 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.765291+0000", 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4576000000000001e-05 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970157+0000", 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20486607000000001 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970242+0000", 2024-04-18T23:11:13.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4662999999999999e-05 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/31 ECSubWrite(tid=59, reqid=client.4568.0:197, at_version=36'22, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.762534+0000", 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.655354132999999, 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.208314309, 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762534+0000", 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762534+0000", 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762535+0000", 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.223e-06 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762562+0000", 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6974999999999998e-05 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762583+0000", 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1126000000000001e-05 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762595+0000", 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2140000000000001e-05 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.766273+0000", 2024-04-18T23:11:13.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036778969999999999 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.766297+0000", 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3289e-05 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970782+0000", 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20448570899999999 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970848+0000", 2024-04-18T23:11:13.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5950000000000004e-05 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/31 ECSubReadReply(tid=16, attrs_read=0))", 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.768604+0000", 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.649283533, 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20125760600000001, 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.768604+0000", 2024-04-18T23:11:13.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.768604+0000", 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.768606+0000", 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.697e-06 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.768984+0000", 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00037785500000000002 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769016+0000", 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1942e-05 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769042+0000", 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6217999999999999e-05 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.969785+0000", 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.200742384 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.969862+0000", 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7509999999999995e-05 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/31 ECSubReadReply(tid=17, attrs_read=0))", 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.769311+0000", 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.648576739999999, 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20064516099999999, 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.480 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769311+0000", 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769311+0000", 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769311+0000", 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769311+0000", 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769342+0000", 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0304999999999999e-05 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769358+0000", 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6028999999999998e-05 2024-04-18T23:11:13.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.969901+0000", 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20054298000000001 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.969956+0000", 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5847000000000002e-05 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/31 ECSubReadReply(tid=17, attrs_read=0))", 2024-04-18T23:11:13.483 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.769453+0000", 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.64843458, 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20158026700000001, 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769453+0000", 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769453+0000", 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769455+0000", 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.31e-06 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769746+0000", 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029135000000000001 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.769767+0000", 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1259000000000001e-05 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.869874+0000", 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100106362 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970924+0000", 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101050294 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.971034+0000", 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010969200000000001 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/31 ECSubWrite(tid=60, reqid=client.4568.0:198, at_version=36'23, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.870210+0000", 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.547677709, 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103328453, 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870210+0000", 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870210+0000", 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870211+0000", 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3699999999999999e-07 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870250+0000", 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8971999999999999e-05 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870280+0000", 2024-04-18T23:11:13.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9832000000000002e-05 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870302+0000", 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1602000000000001e-05 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970337+0000", 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100035279 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970360+0000", 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3136e-05 2024-04-18T23:11:13.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.973463+0000", 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031034320000000002 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.973539+0000", 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5363000000000003e-05 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.490 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/31 ECSubWrite(tid=62, reqid=client.4568.0:212, at_version=36'25, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.870428+0000", 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.547459978999999, 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10318717600000001, 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870428+0000", 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870428+0000", 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870429+0000", 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5400000000000001e-07 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870515+0000", 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6266000000000004e-05 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870537+0000", 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2019e-05 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.870555+0000", 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7555e-05 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.971105+0000", 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100550271 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.971131+0000", 2024-04-18T23:11:13.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5803e-05 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.973560+0000", 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024290269999999998 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.973615+0000", 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5581000000000002e-05 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.494 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.fs0 36/31 ECSubReadReply(tid=17, attrs_read=0))", 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.940750+0000", 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 12.477138074999999, 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13052601699999999, 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.940750+0000", 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.940750+0000", 2024-04-18T23:11:13.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.940751+0000", 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1699999999999997e-07 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.940994+0000", 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00024318199999999999 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.941010+0000", 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6441999999999999e-05 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.941021+0000", 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0416e-05 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.070090+0000", 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.12906909799999999 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.071276+0000", 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011860620000000001 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.497 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:13.498 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:13.498 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:13.499 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:13.499 INFO:tasks.rados.rados.0.smithi142.stdout:134: finishing rollback tid 1 to smithi14230322-27 2024-04-18T23:11:13.499 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:13.500 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:11:13.500 INFO:tasks.rados.rados.0.smithi142.stdout:134: finishing rollback tid 2 to smithi14230322-27 2024-04-18T23:11:13.500 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 27 v 33 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-04-18T23:11:13.500 INFO:tasks.rados.rados.0.smithi142.stdout:134: done (7 left) 2024-04-18T23:11:13.500 INFO:tasks.rados.rados.0.smithi142.stdout:142: done (6 left) 2024-04-18T23:11:13.501 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 23 current snap is 8 2024-04-18T23:11:13.501 INFO:tasks.rados.rados.0.smithi142.stdout:143: seq_num 69 ranges {1277952=786432,2064384=376832} 2024-04-18T23:11:13.501 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:13.500+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:13.513 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:13.514 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:13.526 INFO:tasks.rados.rados.0.smithi142.stdout:143: writing smithi14230322-23 from 1277952 to 2064384 tid 1 2024-04-18T23:11:13.531 INFO:tasks.rados.rados.0.smithi142.stdout:143: writing smithi14230322-23 from 2064384 to 2441216 tid 2 2024-04-18T23:11:13.531 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 9 current snap is 8 2024-04-18T23:11:13.531 INFO:tasks.rados.rados.0.smithi142.stdout:144: seq_num 70 ranges {1163264=802816,1966080=376832} 2024-04-18T23:11:13.542 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:13.542 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:13.556 INFO:tasks.rados.rados.0.smithi142.stdout:144: writing smithi14230322-9 from 1163264 to 1966080 tid 1 2024-04-18T23:11:13.560 INFO:tasks.rados.rados.0.smithi142.stdout:144: writing smithi14230322-9 from 1966080 to 2342912 tid 2 2024-04-18T23:11:13.560 INFO:tasks.rados.rados.0.smithi142.stdout:145: rollback oid 26 current snap is 8 2024-04-18T23:11:13.561 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 26 to 2 2024-04-18T23:11:13.561 INFO:tasks.rados.rados.0.smithi142.stdout:146: snap_create 2024-04-18T23:11:13.564 INFO:tasks.rados.rados.0.smithi142.stdout:141: finishing copy_from racing read to smithi14230322-20 2024-04-18T23:11:13.574 INFO:tasks.rados.rados.0.smithi142.stdout:138: finishing write tid 1 to smithi14230322-38 2024-04-18T23:11:13.574 INFO:tasks.rados.rados.0.smithi142.stdout:138: finishing write tid 2 to smithi14230322-38 2024-04-18T23:11:13.599 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:13.598+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:13.649 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:13.650 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:13.650 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:13.650 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:13.664 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-18T23:11:13.671 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:13.671 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:13.700 INFO:tasks.rados.rados.0.smithi142.stdout:137: finishing copy_from to smithi14230322-12 2024-04-18T23:11:13.700 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 12 v 4 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-18T23:11:13.701 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:13.698+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:13.703 INFO:tasks.rados.rados.0.smithi142.stdout:140: finishing copy_from to smithi14230322-31 2024-04-18T23:11:13.703 INFO:tasks.rados.rados.0.smithi142.stdout:140: got expected ENOENT (src dne) 2024-04-18T23:11:13.703 INFO:tasks.rados.rados.0.smithi142.stdout:140: finishing copy_from racing read to smithi14230322-31 2024-04-18T23:11:13.705 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:13.705 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:13.732 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:13.732 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:13.769 INFO:tasks.rados.rados.0.smithi142.stdout:141: finishing copy_from to smithi14230322-20 2024-04-18T23:11:13.769 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 20 v 27 (ObjNum 61 snap 6 seq_num 61) dirty exists 2024-04-18T23:11:13.771 INFO:tasks.rados.rados.0.smithi142.stdout:138: finishing write tid 3 to smithi14230322-38 2024-04-18T23:11:13.775 INFO:tasks.rados.rados.0.smithi142.stdout:139: finishing write tid 1 to smithi14230322-35 2024-04-18T23:11:13.802 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:13.800+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:13.821 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:13.821 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:13.821 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:13.821 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:13.821 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:13.834 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-18T23:11:13.876 INFO:tasks.rados.rados.0.smithi142.stdout:139: finishing write tid 2 to smithi14230322-35 2024-04-18T23:11:13.878 INFO:tasks.rados.rados.0.smithi142.stdout:132: finishing write tid 4 to smithi14230322-49 2024-04-18T23:11:13.878 INFO:tasks.rados.rados.0.smithi142.stdout:132: oid 49 updating version 0 to 35 2024-04-18T23:11:13.878 INFO:tasks.rados.rados.0.smithi142.stdout:132: oid 49 version 35 is already newer than 34 2024-04-18T23:11:13.878 INFO:tasks.rados.rados.0.smithi142.stdout:132: oid 49 updating version 35 to 36 2024-04-18T23:11:13.878 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 49 v 36 (ObjNum 64 snap 7 seq_num 64) dirty exists 2024-04-18T23:11:13.878 INFO:tasks.rados.rados.0.smithi142.stdout:132: left oid 49 (ObjNum 64 snap 7 seq_num 64) 2024-04-18T23:11:13.903 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:13.902+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:13.945 INFO:tasks.rados.rados.0.smithi142.stdout:144: finishing write tid 1 to smithi14230322-9 2024-04-18T23:11:13.953 INFO:tasks.rados.rados.0.smithi142.stdout:144: finishing write tid 2 to smithi14230322-9 2024-04-18T23:11:13.999 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:13.999 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:13.999 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:13.999 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:13.999 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:13.999 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:74 3.5s0 3:a09a6894:::smithi14230322-25:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:57.251310+0000", 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.746644642, 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29140727700000002, 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 74 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251310+0000", 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251310+0000", 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251311+0000", 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2160000000000001e-06 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251322+0000", 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0946e-05 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251335+0000", 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2826000000000001e-05 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.251345+0000", 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7499999999999998e-06 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.351471+0000", 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10012691 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.351554+0000", 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2670000000000006e-05 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.351932+0000", 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00037827 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:14.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.352496+0000", 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00056402799999999995 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.542650+0000", 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19015369900000001 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:57.542717+0000", 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6962e-05 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:276 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~770048,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.580723+0000", 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.4172314510000001, 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40792810699999998, 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 276 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.580723+0000", 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.580723+0000", 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.580727+0000", 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3789999999999999e-06 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.580744+0000", 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7122999999999999e-05 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.580830+0000", 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5710000000000004e-05 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.580886+0000", 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5733999999999999e-05 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581070+0000", 2024-04-18T23:11:14.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018421999999999999 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581096+0000", 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5851999999999999e-05 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.684805+0000", 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103709183 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.684977+0000", 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000171631 2024-04-18T23:11:14.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.988651+0000", 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30367427499999999 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:277 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~770048,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.580938+0000", 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.4170167400000002, 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40880871800000002, 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.009 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 277 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.580938+0000", 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.010 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.580938+0000", 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.580939+0000", 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7799999999999999e-06 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581005+0000", 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5296999999999995e-05 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581036+0000", 2024-04-18T23:11:14.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1322000000000003e-05 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581056+0000", 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9660000000000002e-05 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581245+0000", 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018918900000000001 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581263+0000", 2024-04-18T23:11:14.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7785e-05 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.685711+0000", 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104447902 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.685870+0000", 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015915999999999999 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.989746+0000", 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30387662300000001 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.013 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:278 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~770048,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.581088+0000", 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.4168667959999999, 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50991812599999997, 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 278 2024-04-18T23:11:14.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581088+0000", 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581088+0000", 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581089+0000", 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6160000000000001e-06 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581099+0000", 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6169999999999996e-06 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581120+0000", 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0766000000000001e-05 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581137+0000", 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6959000000000001e-05 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581341+0000", 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000204592 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.581352+0000", 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1086e-05 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.686425+0000", 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10507272300000001 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.686561+0000", 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013549499999999999 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.091006+0000", 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40444527200000002 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.5s0 42/31 ECSubReadReply(tid=28, attrs_read=0))", 2024-04-18T23:11:14.018 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.786470+0000", 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.2114842670000003, 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.202218171, 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.786470+0000", 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.786470+0000", 2024-04-18T23:11:14.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.786472+0000", 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4789999999999999e-06 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.787080+0000", 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00060817300000000005 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.787108+0000", 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7915999999999998e-05 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.887199+0000", 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10009117100000001 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.987706+0000", 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100507228 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.988688+0000", 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00098220399999999998 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.5s0 42/31 ECSubReadReply(tid=30, attrs_read=0))", 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.786721+0000", 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.2112330309999999, 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20078680500000001, 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.022 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.786721+0000", 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.786721+0000", 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.786723+0000", 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.513e-06 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.787041+0000", 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000317888 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.787063+0000", 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2056e-05 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.887161+0000", 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10009841799999999 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.987429+0000", 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10026773 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.987508+0000", 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9200000000000001e-05 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.5s0 42/31 ECSubReadReply(tid=30, attrs_read=0))", 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.887813+0000", 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.1101417319999998, 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20323523299999999, 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.887813+0000", 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.887813+0000", 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.887814+0000", 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7000000000000003e-07 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.888153+0000", 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00033900299999999999 2024-04-18T23:11:14.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.888180+0000", 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7059e-05 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.888196+0000", 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6392000000000002e-05 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.090148+0000", 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.201951988 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.091048+0000", 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00089982099999999998 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 42/31 ECSubWriteReply(tid=31, last_complete=42'17, committed=1, applied=1))", 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.888934+0000", 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.1090202769999999, 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20230141099999999, 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.029 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.888934+0000", 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.888934+0000", 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.888936+0000", 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.567e-06 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.888947+0000", 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0986e-05 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889198+0000", 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025123399999999997 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889219+0000", 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1155999999999999e-05 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.091172+0000", 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20195270400000001 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.091236+0000", 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3763999999999995e-05 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 42/31 ECSubWriteReply(tid=31, last_complete=42'17, committed=1, applied=1))", 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.889583+0000", 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.1083717059999998, 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20180457099999999, 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889583+0000", 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889583+0000", 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889585+0000", 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9199999999999998e-06 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889596+0000", 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1567e-05 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889626+0000", 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9898000000000001e-05 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889640+0000", 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4346e-05 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.091337+0000", 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20169632800000001 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.091387+0000", 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0512e-05 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 42/31 ECSubWriteReply(tid=31, last_complete=42'17, committed=1, applied=1))", 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.889738+0000", 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.108216219, 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30188071500000002, 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.036 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889738+0000", 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889738+0000", 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889739+0000", 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.017e-06 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889754+0000", 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4286e-05 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889772+0000", 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8496e-05 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889784+0000", 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1483e-05 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.191585+0000", 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.301801245 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.191619+0000", 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4187999999999999e-05 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 42/31 ECSubWriteReply(tid=31, last_complete=42'17, committed=1, applied=1))", 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.889819+0000", 2024-04-18T23:11:14.039 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.108135248, 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30187062799999997, 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889819+0000", 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889819+0000", 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889820+0000", 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889830+0000", 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9960000000000005e-06 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889845+0000", 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5328999999999998e-05 2024-04-18T23:11:14.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.889854+0000", 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4929999999999998e-06 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.191662+0000", 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30180853200000002 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.191690+0000", 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7557999999999999e-05 2024-04-18T23:11:14.042 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 42/31 ECSubWriteReply(tid=31, last_complete=42'17, committed=1, applied=1))", 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.890417+0000", 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.1075376859999997, 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30228021999999999, 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.890417+0000", 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.890417+0000", 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.890418+0000", 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.404e-06 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.890426+0000", 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5830000000000001e-06 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.890449+0000", 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2765999999999999e-05 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.890461+0000", 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2926e-05 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.191762+0000", 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30130044099999997 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.192697+0000", 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00093510000000000002 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s4 42/31 ECSubWrite(tid=61, reqid=osd.14.0:61, at_version=42'25, trim_to=0'0, roll_forward_to=40'23))", 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:06.959555+0000", 2024-04-18T23:11:14.046 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.0383990389999997, 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.201967918, 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959555+0000", 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959555+0000", 2024-04-18T23:11:14.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959556+0000", 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1399999999999996e-07 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959589+0000", 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2660999999999998e-05 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:06.959604+0000", 2024-04-18T23:11:14.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5653000000000001e-05 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.059767+0000", 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100162558 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.159976+0000", 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10020860600000001 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.159991+0000", 2024-04-18T23:11:14.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5407e-05 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.161445+0000", 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014541879999999999 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.161523+0000", 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8131000000000001e-05 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:289 3.5s0 3:a09a6894:::smithi14230322-25:head [append 0~622592] snapc 6=[6,4] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:07.676664+0000", 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.3212899670000002, 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31561669399999998, 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 289 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.676664+0000", 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.676664+0000", 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.676667+0000", 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8030000000000001e-06 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.677410+0000", 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00074257100000000003 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.677445+0000", 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4938999999999999e-05 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.677462+0000", 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7181999999999999e-05 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.781464+0000", 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104001576 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.781639+0000", 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017536599999999999 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.783798+0000", 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021594409999999998 2024-04-18T23:11:14.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.891785+0000", 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.107986627 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.992070+0000", 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100284624 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.992281+0000", 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021156500000000001 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.055 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:290 3.5s0 3:a09a6894:::smithi14230322-25:head [append 0~376832] snapc 6=[6,4] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:07.683721+0000", 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.3142331509999998, 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40938192600000001, 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 290 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.056 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.683721+0000", 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.683721+0000", 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.683724+0000", 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6070000000000002e-06 2024-04-18T23:11:14.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.684261+0000", 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00053714500000000005 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.684295+0000", 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4005999999999999e-05 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.684323+0000", 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7512e-05 2024-04-18T23:11:14.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.787004+0000", 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10268118499999999 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.787170+0000", 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000166229 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.788435+0000", 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00126483 2024-04-18T23:11:14.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.891789+0000", 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103354472 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:08.092890+0000", 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20110109200000001 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:08.093103+0000", 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021284799999999999 2024-04-18T23:11:14.060 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:291 3.5s0 3:a09a6894:::smithi14230322-25:head [setxattr _header (56) in=7b] snapc 6=[6,4] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:07.684364+0000", 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.3135901390000004, 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40946769199999999, 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:14.061 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 291 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.684364+0000", 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.684364+0000", 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.684366+0000", 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6729999999999999e-06 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.684376+0000", 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.696e-06 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.684397+0000", 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1386000000000002e-05 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.684415+0000", 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7853e-05 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.789358+0000", 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104943068 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.789638+0000", 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028029800000000001 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.790915+0000", 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012769330000000001 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.891950+0000", 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10103497 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.065 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:08.093578+0000", 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.201627642 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:08.093832+0000", 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025417300000000001 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.6s3 47/31 ECSubWrite(tid=20, reqid=client.4568.0:346, at_version=47'14, trim_to=0'0, roll_forward_to=43'12))", 2024-04-18T23:11:14.066 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:13.299502+0000", 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.69845276099999998, 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20311004599999999, 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.299502+0000", 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.299502+0000", 2024-04-18T23:11:14.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.299502+0000", 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8800000000000003e-07 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.299705+0000", 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020306400000000001 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.299718+0000", 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3057999999999999e-05 2024-04-18T23:11:14.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.299729+0000", 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0645999999999999e-05 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.500045+0000", 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.200315985 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.500065+0000", 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0528000000000001e-05 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.502504+0000", 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002438871 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.502612+0000", 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010740599999999999 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.070 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s4 48/31 ECSubWrite(tid=83, reqid=client.4568.0:369, at_version=48'35, trim_to=0'0, roll_forward_to=47'33))", 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:13.664146+0000", 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.33380867199999997, 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.201279123, 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664146+0000", 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664146+0000", 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664146+0000", 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5400000000000002e-07 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664205+0000", 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8934000000000002e-05 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664226+0000", 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1126000000000001e-05 2024-04-18T23:11:14.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664256+0000", 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0188999999999998e-05 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.764441+0000", 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10018431899999999 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.764448+0000", 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4839999999999998e-06 2024-04-18T23:11:14.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.865394+0000", 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100946103 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.865425+0000", 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0514e-05 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.074 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s4 48/31 ECSubWrite(tid=84, reqid=client.4568.0:370, at_version=48'36, trim_to=0'0, roll_forward_to=47'33))", 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:13.664296+0000", 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.33365819000000002, 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.201227247, 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664296+0000", 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664296+0000", 2024-04-18T23:11:14.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664297+0000", 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1299999999999999e-07 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664343+0000", 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5905000000000002e-05 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664366+0000", 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3163000000000002e-05 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664385+0000", 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9093000000000002e-05 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.864921+0000", 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.200535572 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.864929+0000", 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7909999999999993e-06 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.865493+0000", 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00056379599999999998 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.865523+0000", 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0213999999999999e-05 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s4 48/31 ECSubWrite(tid=85, reqid=client.4568.0:371, at_version=48'37, trim_to=0'0, roll_forward_to=47'33))", 2024-04-18T23:11:14.078 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:13.664918+0000", 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.33303619099999998, 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20081756100000001, 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664918+0000", 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664918+0000", 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:14.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664924+0000", 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9569999999999999e-06 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664959+0000", 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4879000000000003e-05 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.664985+0000", 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5439000000000002e-05 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.665005+0000", 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0554999999999999e-05 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.865229+0000", 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.200223656 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.865237+0000", 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7740000000000004e-06 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.865712+0000", 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00047531800000000002 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.865736+0000", 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3983000000000001e-05 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:14.082 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:14.083 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-18T23:11:14.083 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-18T23:11:14.083 INFO:tasks.thrashosds.thrasher:Killing osd 15, live_osds are [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-18T23:11:14.083 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-04-18T23:11:14.083 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:11:14.084 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:14.085 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:14.085 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:14.085 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:14.086 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:14.082+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:14.086 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:11:14.086 INFO:tasks.rados.rados.0.smithi142.stdout:138: finishing write tid 4 to smithi14230322-38 2024-04-18T23:11:14.086 INFO:tasks.rados.rados.0.smithi142.stdout:138: oid 38 updating version 0 to 57 2024-04-18T23:11:14.087 INFO:tasks.rados.rados.0.smithi142.stdout:138: oid 38 updating version 57 to 59 2024-04-18T23:11:14.087 INFO:tasks.rados.rados.0.smithi142.stdout:138: oid 38 version 59 is already newer than 58 2024-04-18T23:11:14.087 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 38 v 59 (ObjNum 67 snap 7 seq_num 67) dirty exists 2024-04-18T23:11:14.087 INFO:tasks.rados.rados.0.smithi142.stdout:138: left oid 38 (ObjNum 67 snap 7 seq_num 67) 2024-04-18T23:11:14.146 INFO:tasks.rados.rados.0.smithi142.stdout:144: finishing write tid 3 to smithi14230322-9 2024-04-18T23:11:14.165 INFO:tasks.ceph.osd.15:Stopped 2024-04-18T23:11:14.165 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.15 2024-04-18T23:11:14.165 INFO:tasks.thrashosds.thrasher:remote for osd 15 is ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:11:14.165 DEBUG:teuthology.orchestra.run.smithi142:> type ceph-objectstore-tool 2024-04-18T23:11:14.166 INFO:tasks.rados.rados.0.smithi142.stdout:145: finishing rollback tid 0 to smithi14230322-26 2024-04-18T23:11:14.167 INFO:tasks.rados.rados.0.smithi142.stdout:145: finishing rollback tid 1 to smithi14230322-26 2024-04-18T23:11:14.170 INFO:teuthology.orchestra.run.smithi142.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-18T23:11:14.171 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-15 --journal-path /var/lib/ceph/osd/ceph-15/journal --op list-pgs 2024-04-18T23:11:14.184 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:14.182+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:14.212 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:14.213 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:14.226 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:11:14.267 INFO:tasks.rados.rados.0.smithi142.stdout:145: finishing rollback tid 2 to smithi14230322-26 2024-04-18T23:11:14.267 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 26 v 37 (ObjNum 19 snap 0 seq_num 19) dirty exists 2024-04-18T23:11:14.280 INFO:tasks.rados.rados.0.smithi142.stdout:139: finishing write tid 3 to smithi14230322-35 2024-04-18T23:11:14.285 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:14.282+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:14.350 INFO:tasks.rados.rados.0.smithi142.stdout:144: finishing write tid 4 to smithi14230322-9 2024-04-18T23:11:14.350 INFO:tasks.rados.rados.0.smithi142.stdout:144: oid 9 updating version 0 to 24 2024-04-18T23:11:14.350 INFO:tasks.rados.rados.0.smithi142.stdout:144: oid 9 updating version 24 to 25 2024-04-18T23:11:14.350 INFO:tasks.rados.rados.0.smithi142.stdout:144: oid 9 updating version 25 to 26 2024-04-18T23:11:14.350 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 9 v 26 (ObjNum 70 snap 8 seq_num 70) dirty exists 2024-04-18T23:11:14.350 INFO:tasks.rados.rados.0.smithi142.stdout:144: left oid 9 (ObjNum 70 snap 8 seq_num 70) 2024-04-18T23:11:14.351 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:14.351 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:14.366 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:11:14.412 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:14.410+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:14.486 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:14.482+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:14.489 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:14.489 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:14.500 INFO:tasks.rados.rados.0.smithi142.stdout:132: done (9 left) 2024-04-18T23:11:14.500 INFO:tasks.rados.rados.0.smithi142.stdout:137: done (8 left) 2024-04-18T23:11:14.500 INFO:tasks.rados.rados.0.smithi142.stdout:138: done (7 left) 2024-04-18T23:11:14.500 INFO:tasks.rados.rados.0.smithi142.stdout:140: done (6 left) 2024-04-18T23:11:14.500 INFO:tasks.rados.rados.0.smithi142.stdout:141: done (5 left) 2024-04-18T23:11:14.500 INFO:tasks.rados.rados.0.smithi142.stdout:144: done (4 left) 2024-04-18T23:11:14.500 INFO:tasks.rados.rados.0.smithi142.stdout:145: done (3 left) 2024-04-18T23:11:14.500 INFO:tasks.rados.rados.0.smithi142.stdout:146: done (2 left) 2024-04-18T23:11:14.500 INFO:tasks.rados.rados.0.smithi142.stdout:147: read oid 22 snap -1 2024-04-18T23:11:14.501 INFO:tasks.rados.rados.0.smithi142.stdout:147: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-18T23:11:14.501 INFO:tasks.rados.rados.0.smithi142.stdout:148: read oid 47 snap -1 2024-04-18T23:11:14.501 INFO:tasks.rados.rados.0.smithi142.stdout:148: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-18T23:11:14.501 INFO:tasks.rados.rados.0.smithi142.stdout:149: snap_create 2024-04-18T23:11:14.505 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:11:14.587 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:14.586+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:14.651 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T23:11:14.668 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:14.668 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:14.683 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:11:14.687 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:14.686+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:14.701 INFO:tasks.rados.rados.0.smithi142.stdout:143: finishing write tid 1 to smithi14230322-23 2024-04-18T23:11:14.701 INFO:tasks.rados.rados.0.smithi142.stdout:143: finishing write tid 2 to smithi14230322-23 2024-04-18T23:11:14.701 INFO:tasks.rados.rados.0.smithi142.stdout:139: finishing write tid 4 to smithi14230322-35 2024-04-18T23:11:14.701 INFO:tasks.rados.rados.0.smithi142.stdout:139: oid 35 updating version 0 to 39 2024-04-18T23:11:14.701 INFO:tasks.rados.rados.0.smithi142.stdout:139: oid 35 updating version 39 to 40 2024-04-18T23:11:14.701 INFO:tasks.rados.rados.0.smithi142.stdout:139: oid 35 version 40 is already newer than 38 2024-04-18T23:11:14.701 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 35 v 40 (ObjNum 68 snap 7 seq_num 68) dirty exists 2024-04-18T23:11:14.701 INFO:tasks.rados.rados.0.smithi142.stdout:139: left oid 35 (ObjNum 68 snap 7 seq_num 68) 2024-04-18T23:11:14.789 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:14.788+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:14.866 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:14.866 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:14.882 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:11:14.890 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:14.888+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:14.907 INFO:tasks.rados.rados.0.smithi142.stdout:148: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-18T23:11:14.991 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:14.986+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:14.999 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:14.999 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:15.003 INFO:tasks.rados.rados.0.smithi142.stdout:143: finishing write tid 3 to smithi14230322-23 2024-04-18T23:11:15.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:11:15.093 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:15.092+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:15.176 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:15.176 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:15.190 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:11:15.193 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:15.194+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:15.206 INFO:tasks.rados.rados.0.smithi142.stdout:143: finishing write tid 4 to smithi14230322-23 2024-04-18T23:11:15.206 INFO:tasks.rados.rados.0.smithi142.stdout:143: oid 23 updating version 0 to 44 2024-04-18T23:11:15.206 INFO:tasks.rados.rados.0.smithi142.stdout:143: oid 23 version 44 is already newer than 43 2024-04-18T23:11:15.206 INFO:tasks.rados.rados.0.smithi142.stdout:143: oid 23 version 44 is already newer than 42 2024-04-18T23:11:15.206 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 23 v 44 (ObjNum 69 snap 8 seq_num 69) dirty exists 2024-04-18T23:11:15.206 INFO:tasks.rados.rados.0.smithi142.stdout:143: left oid 23 (ObjNum 69 snap 8 seq_num 69) 2024-04-18T23:11:15.294 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:15.294+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:15.312 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:15.312 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:15.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:11:15.395 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:15.390+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:15.396 INFO:teuthology.orchestra.run.smithi142.stdout:3.9s4 2024-04-18T23:11:15.396 INFO:teuthology.orchestra.run.smithi142.stdout:3.6s3 2024-04-18T23:11:15.396 INFO:teuthology.orchestra.run.smithi142.stdout:3.5s0 2024-04-18T23:11:15.396 INFO:teuthology.orchestra.run.smithi142.stdout:3.3s3 2024-04-18T23:11:15.396 INFO:teuthology.orchestra.run.smithi142.stdout:3.2s4 2024-04-18T23:11:15.396 INFO:teuthology.orchestra.run.smithi142.stdout:3.0s2 2024-04-18T23:11:15.396 INFO:teuthology.orchestra.run.smithi142.stdout:3.8s5 2024-04-18T23:11:15.396 INFO:teuthology.orchestra.run.smithi142.stdout:2.7 2024-04-18T23:11:15.396 INFO:teuthology.orchestra.run.smithi142.stdout:2.3 2024-04-18T23:11:15.449 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:15.449 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:15.463 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:11:15.496 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:15.496+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:15.499 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-04-18T23:11:15.502 INFO:tasks.rados.rados.0.smithi142.stdout:139: done (4 left) 2024-04-18T23:11:15.502 INFO:tasks.rados.rados.0.smithi142.stdout:143: done (3 left) 2024-04-18T23:11:15.502 INFO:tasks.rados.rados.0.smithi142.stdout:148: done (2 left) 2024-04-18T23:11:15.502 INFO:tasks.rados.rados.0.smithi142.stdout:149: done (1 left) 2024-04-18T23:11:15.502 INFO:tasks.rados.rados.0.smithi142.stdout:150: copy_from oid 25 from oid 46 current snap is 10 2024-04-18T23:11:15.502 INFO:tasks.rados.rados.0.smithi142.stdout:151: delete oid 28 current snap is 10 2024-04-18T23:11:15.597 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:15.594+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:15.624 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:15.624 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:15.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:11:15.724 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:15.722+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:15.799 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:15.794+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:15.804 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:15.804 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:15.819 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:11:15.900 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:15.898+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:15.914 INFO:tasks.rados.rados.0.smithi142.stdout:151: done (2 left) 2024-04-18T23:11:15.914 INFO:tasks.rados.rados.0.smithi142.stdout:152: snap_create 2024-04-18T23:11:15.945 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-15 --journal-path /var/lib/ceph/osd/ceph-15/journal --op export --pgid 3.5s0 --file /var/log/ceph/exp.3.5s0.15 2024-04-18T23:11:15.984 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:15.984 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:15.999 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:11:16.020 INFO:tasks.rados.rados.0.smithi142.stdout:147: expect (ObjNum 21 snap 0 seq_num 21) 2024-04-18T23:11:16.020 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:16.016+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:16.101 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:16.098+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:16.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:16.129 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:16.129 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:16.130 INFO:tasks.rados.rados.0.smithi142.stdout:150: finishing copy_from racing read to smithi14230322-25 2024-04-18T23:11:16.144 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:11:16.202 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:16.200+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:16.230 INFO:tasks.rados.rados.0.smithi142.stdout:150: finishing copy_from to smithi14230322-25 2024-04-18T23:11:16.230 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 25 v 6 (ObjNum 45 snap 0 seq_num 45) dirty exists 2024-04-18T23:11:16.262 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:16.263 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:16.266 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:16.266 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:16.281 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:11:16.281 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:16.282 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:16.301 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:16.301 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:16.302 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:16.302+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:16.321 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:16.321 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:16.341 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:16.341 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:16.361 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:16.361 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:16.382 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:16.382 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:16.403 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:16.403 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:16.403 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:16.398+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:16.421 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:16.421 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:16.440 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:16.441 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:16.455 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.455 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/31 ECSubWrite(tid=2, reqid=client.4568.0:19, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.926928557, 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19915530100000001, 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.456 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528004+0000", 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528243+0000", 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239244 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.457 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528271+0000", 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7535e-05 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528293+0000", 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2395000000000002e-05 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.626035+0000", 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.097742512000000004 2024-04-18T23:11:16.458 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.626059+0000", 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3739000000000001e-05 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727090+0000", 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101030651 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727158+0000", 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8415000000000001e-05 2024-04-18T23:11:16.459 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.917043992, 2024-04-18T23:11:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103172215, 2024-04-18T23:11:16.461 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.462 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537889+0000", 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839999999999999e-06 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537921+0000", 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2360999999999997e-05 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537948+0000", 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6421e-05 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.463 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537963+0000", 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5218999999999999e-05 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638148+0000", 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100185015 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638163+0000", 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4793e-05 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.640970+0000", 2024-04-18T23:11:16.464 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028075439999999999 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.641060+0000", 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9377999999999997e-05 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.2s5 33/31 ECSubWrite(tid=160, reqid=client.4568.0:26, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.623173+0000", 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.831758660999999, 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101822397, 2024-04-18T23:11:16.465 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623173+0000", 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623173+0000", 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.466 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623174+0000", 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.16e-07 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623206+0000", 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2660999999999998e-05 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623229+0000", 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2481999999999999e-05 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.467 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723331+0000", 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010213699999999 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723530+0000", 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019928599999999999 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723553+0000", 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2442e-05 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.468 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724939+0000", 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013867300000000001 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724995+0000", 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5742999999999998e-05 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:5, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:16.469 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.826842782, 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103548802, 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.470 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628091+0000", 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1129999999999999e-06 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628179+0000", 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7818000000000005e-05 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.471 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628219+0000", 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0373000000000001e-05 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628243+0000", 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3549000000000001e-05 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628766+0000", 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052308499999999996 2024-04-18T23:11:16.472 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628791+0000", 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4984999999999999e-05 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731546+0000", 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102755535 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731638+0000", 2024-04-18T23:11:16.473 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1343999999999998e-05 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=2, reqid=client.4568.0:22, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.826630626, 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103381863, 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.474 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628302+0000", 2024-04-18T23:11:16.475 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628476+0000", 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017366199999999999 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628499+0000", 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3025000000000001e-05 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.476 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628519+0000", 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0194999999999999e-05 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629371+0000", 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085246300000000003 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629394+0000", 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3311000000000001e-05 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.477 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731650+0000", 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10225558899999999 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731683+0000", 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2654999999999998e-05 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.478 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=3, reqid=client.4568.0:23, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.825933436, 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103947704, 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.479 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629005+0000", 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7089999999999996e-06 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629040+0000", 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5133999999999999e-05 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.480 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629064+0000", 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4422000000000001e-05 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629079+0000", 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4949e-05 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731181+0000", 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102101339 2024-04-18T23:11:16.481 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731199+0000", 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8175000000000001e-05 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.732877+0000", 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016778050000000001 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.482 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.732946+0000", 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9171000000000006e-05 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 36/31 ECSubWrite(tid=57, reqid=client.4568.0:183, at_version=36'15, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.662500+0000", 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.792431894, 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100992527, 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.483 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662500+0000", 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662500+0000", 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662500+0000", 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7300000000000005e-07 2024-04-18T23:11:16.484 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662522+0000", 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1399e-05 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662534+0000", 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1806e-05 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662543+0000", 2024-04-18T23:11:16.485 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6889999999999999e-06 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.762211+0000", 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.099667800000000001 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.762228+0000", 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7173e-05 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.486 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763390+0000", 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00116117 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.763492+0000", 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102617 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 36/31 ECSubWrite(tid=10, reqid=client.4568.0:176, at_version=0'0, trim_to=0'0, roll_forward_to=36'8))", 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:01.071044+0000", 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.383887954, 2024-04-18T23:11:16.487 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101470028, 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071044+0000", 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071044+0000", 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.488 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071045+0000", 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5209999999999999e-06 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071064+0000", 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.861e-05 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071099+0000", 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5701000000000003e-05 2024-04-18T23:11:16.489 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.171205+0000", 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100105175 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.171312+0000", 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000107008 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.171327+0000", 2024-04-18T23:11:16.490 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5224e-05 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172398+0000", 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010713090000000001 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172514+0000", 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011548 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.491 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 40/31 ECSubWrite(tid=19, reqid=osd.5.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:05.189920+0000", 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.265011489999999, 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101951974, 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189920+0000", 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.492 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189920+0000", 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189923+0000", 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189945+0000", 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2002e-05 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.493 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189977+0000", 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2122000000000002e-05 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.290109+0000", 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100132397 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.290251+0000", 2024-04-18T23:11:16.494 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000141395 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.290277+0000", 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5959e-05 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.291785+0000", 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001507993 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.495 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.291872+0000", 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7425999999999999e-05 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 42/31 ECSubWrite(tid=27, reqid=osd.8.0:27, at_version=42'21, trim_to=0'0, roll_forward_to=40'19))", 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:06.598717+0000", 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.8562149869999995, 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101705305, 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.496 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598717+0000", 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598717+0000", 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598717+0000", 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4299999999999998e-07 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.497 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598760+0000", 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3538000000000002e-05 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598784+0000", 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3618e-05 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598800+0000", 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5489999999999999e-05 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.498 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.699022+0000", 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100222062 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.699037+0000", 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5290000000000001e-05 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.700359+0000", 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001322043 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.499 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.700422+0000", 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2821e-05 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.6s2 42/31 ECSubWrite(tid=17, reqid=osd.0.0:16, at_version=0'0, trim_to=0'0, roll_forward_to=42'10))", 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:06.784649+0000", 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.6702829650000002, 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10152433399999999, 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.500 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.784649+0000", 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.784649+0000", 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.501 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.784650+0000", 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.756e-06 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.784673+0000", 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2938999999999998e-05 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.784712+0000", 2024-04-18T23:11:16.502 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8427999999999999e-05 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.784735+0000", 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3479e-05 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.885039+0000", 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100303419 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.503 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.885053+0000", 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4766000000000001e-05 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.886077+0000", 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001023742 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.886173+0000", 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5804999999999995e-05 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.504 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 42/31 ECSubWrite(tid=87, reqid=osd.4.0:85, at_version=0'0, trim_to=0'0, roll_forward_to=42'27))", 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:06.785569+0000", 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.6693625680000004, 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101991601, 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785569+0000", 2024-04-18T23:11:16.505 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785569+0000", 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785571+0000", 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.506 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785600+0000", 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8812000000000001e-05 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785630+0000", 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0015e-05 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.885735+0000", 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010559199999999 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.885865+0000", 2024-04-18T23:11:16.507 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000130087 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.885914+0000", 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8853999999999997e-05 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.887429+0000", 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001515048 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.887561+0000", 2024-04-18T23:11:16.508 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013133 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 42/31 ECSubWrite(tid=28, reqid=osd.8.0:27, at_version=0'0, trim_to=0'0, roll_forward_to=42'21))", 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:06.888631+0000", 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5663008739999995, 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10187332, 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.509 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888631+0000", 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888631+0000", 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888631+0000", 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000002e-07 2024-04-18T23:11:16.510 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888653+0000", 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1259000000000001e-05 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888684+0000", 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0885e-05 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.988782+0000", 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10009855600000001 2024-04-18T23:11:16.511 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.989001+0000", 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021871600000000001 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.989034+0000", 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3071000000000002e-05 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.990449+0000", 2024-04-18T23:11:16.512 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001415166 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.990504+0000", 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4824000000000002e-05 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 43/31 ECSubWrite(tid=30, reqid=osd.8.0:30, at_version=43'23, trim_to=0'0, roll_forward_to=42'21))", 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:07.609293+0000", 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.8456383439999993, 2024-04-18T23:11:16.513 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10199349200000001, 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609293+0000", 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609293+0000", 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.514 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609294+0000", 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4000000000000002e-07 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609357+0000", 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3386999999999995e-05 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609390+0000", 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2838e-05 2024-04-18T23:11:16.515 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609416+0000", 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5799000000000002e-05 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.709727+0000", 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100311764 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.709776+0000", 2024-04-18T23:11:16.516 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8198000000000002e-05 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.711192+0000", 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00141602 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.711287+0000", 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4945999999999995e-05 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.517 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.bs2 46/31 ECSubRead(tid=109, to_read={3:d447fef0:::smithi14230322-49:head=0,204800,0}, subchunks={3:d447fef0:::smithi14230322-49:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:10.916304+0000", 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.5386271420000002, 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100661348, 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:10.916304+0000", 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.518 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:10.916304+0000", 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:10.916306+0000", 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:10.916322+0000", 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6028999999999998e-05 2024-04-18T23:11:16.519 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:10.916355+0000", 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3324999999999997e-05 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:10.916374+0000", 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8406e-05 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.016660+0000", 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100286668 2024-04-18T23:11:16.520 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.016966+0000", 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00030541299999999998 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 46/31 ECSubWrite(tid=112, reqid=client.4568.0:326, at_version=46'31, trim_to=0'0, roll_forward_to=42'27))", 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:11.434154+0000", 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.0207778789999997, 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101536472, 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.521 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.434154+0000", 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.434154+0000", 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.522 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.434155+0000", 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6559999999999999e-06 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.434237+0000", 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1829999999999994e-05 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.434267+0000", 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9402000000000001e-05 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.523 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.534375+0000", 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100108866 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.534504+0000", 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012833000000000001 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.534521+0000", 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7606999999999999e-05 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.524 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.535613+0000", 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001092148 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.535690+0000", 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6632999999999996e-05 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 47/31 ECSubWrite(tid=22, reqid=osd.5.0:22, at_version=47'16, trim_to=0'0, roll_forward_to=46'14))", 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.623975+0000", 2024-04-18T23:11:16.525 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.83095652, 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102688324, 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.623975+0000", 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.623975+0000", 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.623976+0000", 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9299999999999997e-07 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.624030+0000", 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4165000000000001e-05 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.624064+0000", 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4610999999999998e-05 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.724208+0000", 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10014355799999999 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.724385+0000", 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000177123 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.724414+0000", 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8600999999999999e-05 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.726569+0000", 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002155751 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.726663+0000", 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3821999999999997e-05 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 47/31 ECSubWrite(tid=81, reqid=client.4568.0:343, at_version=47'33, trim_to=0'0, roll_forward_to=45'29))", 2024-04-18T23:11:16.529 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.931370+0000", 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5235615849999999, 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101574505, 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931370+0000", 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931370+0000", 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931371+0000", 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5570000000000001e-06 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931416+0000", 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4580999999999999e-05 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931452+0000", 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5549000000000001e-05 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931476+0000", 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4532000000000002e-05 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031741+0000", 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10026481700000001 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031756+0000", 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4603000000000001e-05 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.032868+0000", 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001112863 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.032944+0000", 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6002999999999997e-05 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.533 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 47/31 ECSubWrite(tid=116, reqid=client.4568.0:335, at_version=47'35, trim_to=0'0, roll_forward_to=47'33))", 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.935986+0000", 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.5189450340000001, 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102410364, 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.935986+0000", 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.935986+0000", 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.935988+0000", 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8729999999999999e-06 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936143+0000", 2024-04-18T23:11:16.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015484299999999999 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936174+0000", 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0571999999999998e-05 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036325+0000", 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10015107400000001 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036499+0000", 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017410200000000001 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036514+0000", 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4586000000000001e-05 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038304+0000", 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017909309999999999 2024-04-18T23:11:16.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038397+0000", 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2383e-05 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 47/31 ECSubWrite(tid=117, reqid=client.4568.0:336, at_version=47'36, trim_to=0'0, roll_forward_to=47'33))", 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:13.036385+0000", 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 3.4185466419999999, 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10165819700000001, 2024-04-18T23:11:16.538 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036385+0000", 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036385+0000", 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036386+0000", 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2270000000000001e-06 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036427+0000", 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1270999999999998e-05 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036453+0000", 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5758999999999999e-05 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.136595+0000", 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100142075 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.136751+0000", 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000155623 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.136779+0000", 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8322000000000001e-05 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.137961+0000", 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011816909999999999 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.138043+0000", 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2229000000000002e-05 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.542 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:16.543 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:16.544 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:16.544 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:16.545 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:16.545 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:11:16.546 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:16.546 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:16.546 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:16.546 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:16.547 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:16.547 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T23:11:16.548 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:16.548 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENXIO: problem getting command descriptions from osd.15 2024-04-18T23:11:16.548 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T23:11:16.548 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:16.542+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:16.644 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:16.640+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:16.655 INFO:tasks.rados.rados.0.smithi142.stdout:147: done (2 left) 2024-04-18T23:11:16.655 INFO:tasks.rados.rados.0.smithi142.stdout:150: done (1 left) 2024-04-18T23:11:16.655 INFO:tasks.rados.rados.0.smithi142.stdout:152: done (0 left) 2024-04-18T23:11:16.655 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 43 current snap is 11 2024-04-18T23:11:16.655 INFO:tasks.rados.rados.0.smithi142.stdout:153: seq_num 72 ranges {720896=442368,1163264=720896,1884160=49152} 2024-04-18T23:11:16.662 INFO:tasks.rados.rados.0.smithi142.stdout:153: writing smithi14230322-43 from 720896 to 1163264 tid 1 2024-04-18T23:11:16.666 INFO:tasks.rados.rados.0.smithi142.stdout:153: writing smithi14230322-43 from 1163264 to 1884160 tid 2 2024-04-18T23:11:16.666 INFO:tasks.rados.rados.0.smithi142.stdout:153: writing smithi14230322-43 from 1884160 to 1933312 tid 3 2024-04-18T23:11:16.667 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 10 current snap is 11 2024-04-18T23:11:16.667 INFO:tasks.rados.rados.0.smithi142.stdout:154: seq_num 73 ranges {163840=458752,622592=475136,1097728=294912} 2024-04-18T23:11:16.670 INFO:tasks.rados.rados.0.smithi142.stdout:154: writing smithi14230322-10 from 163840 to 622592 tid 1 2024-04-18T23:11:16.672 INFO:tasks.rados.rados.0.smithi142.stdout:154: writing smithi14230322-10 from 622592 to 1097728 tid 2 2024-04-18T23:11:16.674 INFO:tasks.rados.rados.0.smithi142.stdout:154: writing smithi14230322-10 from 1097728 to 1392640 tid 3 2024-04-18T23:11:16.674 INFO:tasks.rados.rados.0.smithi142.stdout:155: snap_create 2024-04-18T23:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:16.677 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:16.691 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:11:16.745 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:16.742+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:16.813 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:16.813 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:16.813 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:16.827 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:11:16.874 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:16.876+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:16.946 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:16.948+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:16.951 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 33/31 ECSubWrite(tid=1, reqid=client.4568.0:16, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.520061+0000", 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.431263328, 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10251751100000001, 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520061+0000", 2024-04-18T23:11:16.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520061+0000", 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520063+0000", 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.95e-06 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520209+0000", 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014583700000000001 2024-04-18T23:11:16.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520243+0000", 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4334e-05 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520266+0000", 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3118000000000001e-05 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.620586+0000", 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100320109 2024-04-18T23:11:16.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.620600+0000", 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3453e-05 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622489+0000", 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018891730000000001 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622578+0000", 2024-04-18T23:11:16.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9536999999999993e-05 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 33/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.623740+0000", 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.327584744999999, 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100701047, 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623740+0000", 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623740+0000", 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623752+0000", 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2886e-05 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623765+0000", 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3046e-05 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623796+0000", 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0793999999999997e-05 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723901+0000", 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010519 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724256+0000", 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000354149 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724441+0000", 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018498200000000001 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:16.959 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.724526+0000", 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.226798640999998, 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10203172000000001, 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724526+0000", 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724526+0000", 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724527+0000", 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1799999999999999e-06 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724534+0000", 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3830000000000003e-06 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724556+0000", 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2317999999999999e-05 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724575+0000", 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8425999999999999e-05 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826497+0000", 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10192208699999999 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826557+0000", 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.0325999999999998e-05 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 34/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.725568+0000", 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.225756370999999, 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101108719, 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.963 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725568+0000", 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725568+0000", 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725570+0000", 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7e-06 2024-04-18T23:11:16.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725579+0000", 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5959999999999993e-06 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725607+0000", 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7925000000000001e-05 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.725623+0000", 2024-04-18T23:11:16.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5719e-05 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826642+0000", 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101018832 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.826677+0000", 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4947000000000003e-05 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:176 3.8s0 3:1090b2d1:::smithi14230322-15:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.411688572999999, 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.53110666500000003, 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 176 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539638+0000", 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3829999999999999e-06 2024-04-18T23:11:16.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539651+0000", 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2549e-05 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539675+0000", 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4413000000000002e-05 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539689+0000", 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4161999999999999e-05 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539831+0000", 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014226500000000001 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op must wait for map", 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539865+0000", 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3185000000000002e-05 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644415+0000", 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10455017599999999 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644498+0000", 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3568999999999999e-05 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.865827+0000", 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.22132859799999999 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.868846+0000", 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030189309999999999 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070576+0000", 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20172967 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070742+0000", 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016676399999999999 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:177 3.8s0 3:1090b2d1:::smithi14230322-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:16.972 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.411605941000001, 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32703084799999999, 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 177 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539719+0000", 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.37e-07 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539723+0000", 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1620000000000001e-06 2024-04-18T23:11:16.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539733+0000", 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4630000000000007e-06 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539739+0000", 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7880000000000001e-06 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540029+0000", 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029001100000000001 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.975 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540050+0000", 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1033e-05 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644678+0000", 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10462763999999999 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644714+0000", 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6202999999999999e-05 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866546+0000", 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.22183165399999999 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866650+0000", 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104465 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866749+0000", 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8791999999999999e-05 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:207 3.8s0 3:1c19a06b:::smithi14230322-28:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.389989774, 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40709379200000001, 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 207 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561336+0000", 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.638e-06 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561354+0000", 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7413000000000001e-05 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561372+0000", 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8003e-05 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.980 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561383+0000", 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1425e-05 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561495+0000", 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111661 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561506+0000", 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.113e-05 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644748+0000", 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.083242308000000001 2024-04-18T23:11:16.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644807+0000", 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8909999999999997e-05 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.854271+0000", 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.209463753 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.856288+0000", 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020173909999999999 2024-04-18T23:11:16.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.968263+0000", 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.111975256 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.968428+0000", 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016490400000000001 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.983 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:208 3.8s0 3:1c19a06b:::smithi14230322-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.389916684999999, 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29446849200000003, 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 208 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.984 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561411+0000", 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9179999999999998e-06 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561421+0000", 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4599999999999992e-06 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561427+0000", 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561582+0000", 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000154652 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561592+0000", 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.004e-05 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644937+0000", 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.083345591999999996 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644974+0000", 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6835000000000002e-05 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855704+0000", 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.21072980599999999 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855787+0000", 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3442000000000005e-05 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855876+0000", 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8660999999999997e-05 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/31 ECSubWrite(tid=60, reqid=client.4568.0:198, at_version=36'23, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.764060+0000", 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.187264523, 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101243635, 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764060+0000", 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764060+0000", 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764061+0000", 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0729999999999999e-06 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764087+0000", 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6444999999999998e-05 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.764104+0000", 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6622000000000001e-05 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.864187+0000", 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10008339199999999 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.864266+0000", 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9182000000000002e-05 2024-04-18T23:11:16.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.864274+0000", 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4000000000000003e-06 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.865258+0000", 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000983755 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.865303+0000", 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5766000000000002e-05 2024-04-18T23:11:16.992 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 36/31 ECSubWrite(tid=62, reqid=client.4568.0:212, at_version=36'25, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.969268+0000", 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.982056533, 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101963971, 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.969268+0000", 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.969268+0000", 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.969268+0000", 2024-04-18T23:11:16.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4600000000000004e-07 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.969345+0000", 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6153000000000001e-05 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.969363+0000", 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8712e-05 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.969376+0000", 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2533e-05 2024-04-18T23:11:16.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.069561+0000", 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100185306 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.069577+0000", 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5866e-05 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071194+0000", 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016167829999999999 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071232+0000", 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7972000000000002e-05 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 42/31 ECSubWrite(tid=88, reqid=osd.4.0:86, at_version=0'0, trim_to=0'0, roll_forward_to=42'39))", 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:06.887161+0000", 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.064163193000001, 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10148993000000001, 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:16.997 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.887161+0000", 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.887161+0000", 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.887163+0000", 2024-04-18T23:11:16.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0099999999999998e-06 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.887192+0000", 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8799e-05 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.887227+0000", 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4811999999999998e-05 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.987323+0000", 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10009628600000001 2024-04-18T23:11:16.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.987458+0000", 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013481999999999999 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.987473+0000", 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5389000000000001e-05 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.988559+0000", 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010859349999999999 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.988651+0000", 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1879000000000006e-05 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 42/31 ECSubWrite(tid=90, reqid=client.4568.0:279, at_version=0'0, trim_to=0'0, roll_forward_to=42'41))", 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:07.396027+0000", 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.5552972829999998, 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20190517899999999, 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.001 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396027+0000", 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396027+0000", 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396029+0000", 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.553e-06 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396049+0000", 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0109e-05 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396078+0000", 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9102e-05 2024-04-18T23:11:17.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396096+0000", 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8301999999999999e-05 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.596441+0000", 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20034464299999999 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.596477+0000", 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6189000000000002e-05 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.597862+0000", 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013854780000000001 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.597932+0000", 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9802999999999996e-05 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.005 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:331 3.8s0 3:1c19a06b:::smithi14230322-28:head [stat,delete] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.5224369209999997, 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205262474, 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 331 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.006 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428890+0000", 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4830000000000002e-06 2024-04-18T23:11:17.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428901+0000", 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0852999999999999e-05 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428933+0000", 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2234999999999999e-05 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428956+0000", 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3436000000000001e-05 2024-04-18T23:11:17.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.429116+0000", 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015961600000000001 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.429402+0000", 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028557099999999998 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.431719+0000", 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002317117 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.434261+0000", 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025422980000000001 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.633920+0000", 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.199659052 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.634150+0000", 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022981299999999999 2024-04-18T23:11:17.010 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 46/31 ECSubWriteReply(tid=21, last_complete=46'14, committed=1, applied=1))", 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:11.636567+0000", 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3147567870000003, 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19982181299999999, 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636567+0000", 2024-04-18T23:11:17.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636567+0000", 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636569+0000", 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.403e-06 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636578+0000", 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8689999999999993e-06 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636605+0000", 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7056e-05 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636619+0000", 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4269e-05 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.836343+0000", 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19972390100000001 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.836389+0000", 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6315e-05 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.730926+0000", 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2203978019999999, 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20030836399999999, 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730926+0000", 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730926+0000", 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730928+0000", 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.547e-06 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730940+0000", 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2079000000000001e-05 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730975+0000", 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5361999999999998e-05 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730992+0000", 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6426000000000002e-05 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931197+0000", 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.200204826 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931235+0000", 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8124000000000003e-05 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.220249345, 2024-04-18T23:11:17.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30085146299999999, 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731076+0000", 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6599999999999994e-07 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731085+0000", 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6670000000000002e-06 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731102+0000", 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7368999999999999e-05 2024-04-18T23:11:17.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731113+0000", 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1552999999999999e-05 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031861+0000", 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30074756000000002 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031926+0000", 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5347999999999998e-05 2024-04-18T23:11:17.020 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731177+0000", 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2201477089999999, 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20023347999999999, 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731177+0000", 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731177+0000", 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731178+0000", 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4959999999999999e-06 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731186+0000", 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731210+0000", 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3938999999999999e-05 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731223+0000", 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.305e-05 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931361+0000", 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20013773200000001 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931410+0000", 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8937000000000003e-05 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:17.024 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731597+0000", 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2197276239999999, 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.199919018, 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731597+0000", 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731597+0000", 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731598+0000", 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4470000000000001e-06 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731608+0000", 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5089999999999999e-06 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731638+0000", 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0574999999999998e-05 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731655+0000", 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6592999999999999e-05 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931474+0000", 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19981897600000001 2024-04-18T23:11:17.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931516+0000", 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1918000000000001e-05 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731611+0000", 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2197137549999999, 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20007641100000001, 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.028 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731611+0000", 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731611+0000", 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731611+0000", 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0999999999999998e-07 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731625+0000", 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.366e-05 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731642+0000", 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7042e-05 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731658+0000", 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931663+0000", 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20000464400000001 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931687+0000", 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3811999999999998e-05 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.031 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 46/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731775+0000", 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2195493050000001, 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19982924099999999, 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731775+0000", 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731775+0000", 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731776+0000", 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.486e-06 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731784+0000", 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7370000000000006e-06 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731802+0000", 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7492000000000001e-05 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731811+0000", 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.713e-06 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931577+0000", 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19976532799999999 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931604+0000", 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7484999999999999e-05 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.035 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:17.037 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:11:17.079 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:17.080+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:17.141 INFO:teuthology.orchestra.run.smithi142.stderr:Exporting 3.5s0 info 3.5s0( v 44'25 (0'0,44'25] local-lis/les=31/32 n=5 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) 2024-04-18T23:11:17.148 INFO:teuthology.orchestra.run.smithi142.stderr:Read 0#3:a09a6894:::smithi14230322-25:6# 2024-04-18T23:11:17.148 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:17.148+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:17.149 INFO:teuthology.orchestra.run.smithi142.stderr:Read 0#3:a09a6894:::smithi14230322-25:head# 2024-04-18T23:11:17.150 INFO:teuthology.orchestra.run.smithi142.stderr:Read 0#3:a6fe874e:::smithi14230322-33:5# 2024-04-18T23:11:17.151 INFO:teuthology.orchestra.run.smithi142.stderr:Read 0#3:a6fe874e:::smithi14230322-33:head# 2024-04-18T23:11:17.151 INFO:teuthology.orchestra.run.smithi142.stderr:Read 0#3:a832c25e:::smithi14230322-46:head# 2024-04-18T23:11:17.152 INFO:teuthology.orchestra.run.smithi142.stderr:Export successful 2024-04-18T23:11:17.180 INFO:tasks.rados.rados.0.smithi142.stdout:153: finishing write tid 1 to smithi14230322-43 2024-04-18T23:11:17.208 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:17.208 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:17.208 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:17.222 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:11:17.249 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:17.250+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:17.279 INFO:tasks.rados.rados.0.smithi142.stdout:153: finishing write tid 2 to smithi14230322-43 2024-04-18T23:11:17.280 INFO:tasks.rados.rados.0.smithi142.stdout:153: finishing write tid 3 to smithi14230322-43 2024-04-18T23:11:17.280 INFO:tasks.rados.rados.0.smithi142.stdout:153: finishing write tid 4 to smithi14230322-43 2024-04-18T23:11:17.350 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:17.348+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:17.382 INFO:tasks.rados.rados.0.smithi142.stdout:154: finishing write tid 1 to smithi14230322-10 2024-04-18T23:11:17.387 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:17.387 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:17.387 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:17.401 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:11:17.450 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:17.450+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:17.482 INFO:tasks.rados.rados.0.smithi142.stdout:154: finishing write tid 2 to smithi14230322-10 2024-04-18T23:11:17.482 INFO:tasks.rados.rados.0.smithi142.stdout:154: finishing write tid 3 to smithi14230322-10 2024-04-18T23:11:17.483 INFO:tasks.rados.rados.0.smithi142.stdout:154: finishing write tid 4 to smithi14230322-10 2024-04-18T23:11:17.523 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T23:11:17.525 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.1s2 33/31 ECSubWrite(tid=3, reqid=client.4568.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526871+0000", 2024-04-18T23:11:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.999582704000002, 2024-04-18T23:11:17.525 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10188062000000001, 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526871+0000", 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526871+0000", 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.526 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526873+0000", 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526917+0000", 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4326999999999998e-05 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9663000000000003e-05 2024-04-18T23:11:17.527 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526981+0000", 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3915999999999999e-05 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.627401+0000", 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10041953200000001 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.627425+0000", 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4085000000000001e-05 2024-04-18T23:11:17.528 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628646+0000", 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012212010000000001 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628752+0000", 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010604599999999999 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.529 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 34/31 ECSubWrite(tid=0, reqid=client.4568.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.535528+0000", 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.990926166000001, 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.089577750999999997, 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535528+0000", 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.530 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535528+0000", 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535529+0000", 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.22e-06 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535833+0000", 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00030396900000000002 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.531 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535865+0000", 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1563999999999998e-05 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.535881+0000", 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6039e-05 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.536062+0000", 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018088199999999999 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.536086+0000", 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4465e-05 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625023+0000", 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.088936666999999997 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625106+0000", 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2945e-05 2024-04-18T23:11:17.533 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.537968+0000", 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.988485504, 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.087241641999999994, 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537968+0000", 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537968+0000", 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537970+0000", 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.33e-06 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538023+0000", 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3340000000000001e-05 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538048+0000", 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4851999999999999e-05 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538070+0000", 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1658000000000002e-05 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538180+0000", 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000110676 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.538198+0000", 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7261999999999998e-05 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625116+0000", 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.086918529999999994 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.625210+0000", 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3993999999999995e-05 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 34/31 ECSubWrite(tid=9, reqid=client.4568.0:41, at_version=34'4, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.737558+0000", 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.788896199, 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.098370887000000004, 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737558+0000", 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737558+0000", 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737559+0000", 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5069999999999999e-06 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737599+0000", 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0030000000000001e-05 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737634+0000", 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4378000000000002e-05 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.737656+0000", 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2478000000000001e-05 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.834675+0000", 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.097018668000000002 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.834702+0000", 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7625000000000001e-05 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.835833+0000", 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011305740000000001 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.835929+0000", 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5626999999999998e-05 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.542 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.1s2 34/31 ECSubWrite(tid=10, reqid=client.4568.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.825050+0000", 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.701403531, 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10149166599999999, 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825050+0000", 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825050+0000", 2024-04-18T23:11:17.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825052+0000", 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.61e-06 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825085+0000", 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2428000000000002e-05 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825112+0000", 2024-04-18T23:11:17.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7168000000000001e-05 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825133+0000", 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1061999999999999e-05 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.925428+0000", 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100294913 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.925442+0000", 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4528999999999999e-05 2024-04-18T23:11:17.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.926453+0000", 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001010515 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.926542+0000", 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9441e-05 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.1s2 34/31 ECSubRead(tid=11, to_read={3:8af79e0a:::smithi14230322-1:head=0,4096,0}, subchunks={3:8af79e0a:::smithi14230322-1:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.825619+0000", 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.700834665999999, 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10067092800000001, 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825619+0000", 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825619+0000", 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825622+0000", 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2900000000000001e-06 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825640+0000", 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8289000000000001e-05 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825681+0000", 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1066999999999998e-05 2024-04-18T23:11:17.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.825721+0000", 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0244000000000003e-05 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.925824+0000", 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010322000000001 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.926290+0000", 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000465818 2024-04-18T23:11:17.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 36/31 ECSubWrite(tid=54, reqid=client.4568.0:188, at_version=36'19, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.658481+0000", 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.867972902999998, 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10146903, 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.550 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658481+0000", 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658481+0000", 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658481+0000", 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.92e-07 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658577+0000", 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5395000000000004e-05 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658601+0000", 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4672e-05 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.658610+0000", 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8389999999999996e-06 2024-04-18T23:11:17.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.758786+0000", 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100175606 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.758793+0000", 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2710000000000003e-06 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.759907+0000", 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001113916 2024-04-18T23:11:17.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.759950+0000", 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3139000000000002e-05 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 36/31 ECSubWrite(tid=55, reqid=client.4568.0:181, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.660823+0000", 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.865630982999999, 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0088911570000000002, 2024-04-18T23:11:17.554 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.660823+0000", 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.660823+0000", 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.660824+0000", 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.47e-07 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661023+0000", 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019883800000000001 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661044+0000", 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1175999999999999e-05 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661058+0000", 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3872e-05 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662186+0000", 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001128059 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662197+0000", 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1579e-05 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669659+0000", 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0074618050000000002 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669714+0000", 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4880999999999999e-05 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.558 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 36/31 ECSubWrite(tid=56, reqid=client.4568.0:182, at_version=36'14, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.661892+0000", 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.864561924, 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0078648880000000004, 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661892+0000", 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661892+0000", 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.661893+0000", 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.54e-07 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662067+0000", 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017397800000000001 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662085+0000", 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8085e-05 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662103+0000", 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7949e-05 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.663087+0000", 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098454300000000005 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.663098+0000", 2024-04-18T23:11:17.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0443e-05 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669720+0000", 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0066224420000000001 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669757+0000", 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6594000000000002e-05 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 36/31 ECSubWrite(tid=57, reqid=client.4568.0:183, at_version=36'15, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.662696+0000", 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.863757927000002, 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073676760000000001, 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662696+0000", 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662696+0000", 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662697+0000", 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.217e-06 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662718+0000", 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1205000000000001e-05 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662745+0000", 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6580999999999998e-05 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.662759+0000", 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3900000000000001e-05 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669313+0000", 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0065537579999999998 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.669330+0000", 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7736000000000001e-05 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.670027+0000", 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00069616800000000005 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.670064+0000", 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7110999999999998e-05 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.es1 36/31 ECSubRead(tid=15, to_read={3:78aee268:::smithi14230322-13:head=0,53248,0}, subchunks={3:78aee268:::smithi14230322-13:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.770553+0000", 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.755900796999999, 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100377276, 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770553+0000", 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770553+0000", 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770555+0000", 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9740000000000001e-06 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770577+0000", 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1455e-05 2024-04-18T23:11:17.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.770614+0000", 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7800999999999999e-05 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870693+0000", 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100078207 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870835+0000", 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014253700000000001 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870930+0000", 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5302000000000003e-05 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.es1 36/31 ECSubRead(tid=16, to_read={3:7a4badf6:::smithi14230322-17:head=0,69632,0}, subchunks={3:7a4badf6:::smithi14230322-17:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.870730+0000", 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.655723966, 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20070310299999999, 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.570 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870730+0000", 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870730+0000", 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870731+0000", 2024-04-18T23:11:17.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4099999999999998e-07 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870743+0000", 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2034e-05 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870760+0000", 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6956000000000002e-05 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870776+0000", 2024-04-18T23:11:17.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6276000000000001e-05 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070936+0000", 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20016004100000001 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071433+0000", 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049705500000000004 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 36/31 ECSubWrite(tid=65, reqid=client.4568.0:212, at_version=0'0, trim_to=0'0, roll_forward_to=36'25))", 2024-04-18T23:11:17.573 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:01.172511+0000", 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.353943436000002, 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101623536, 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172511+0000", 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172511+0000", 2024-04-18T23:11:17.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172513+0000", 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4329999999999998e-06 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172544+0000", 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1068000000000001e-05 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172578+0000", 2024-04-18T23:11:17.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4155000000000001e-05 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.172596+0000", 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7325000000000001e-05 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.272858+0000", 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10026249199999999 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.272887+0000", 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9306e-05 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.274061+0000", 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011737410000000001 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.274134+0000", 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3015999999999993e-05 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.577 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 36/31 ECSubWrite(tid=67, reqid=client.4568.0:189, at_version=0'0, trim_to=0'0, roll_forward_to=36'17))", 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:01.472952+0000", 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 16.053502047999999, 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101289249, 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.472952+0000", 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.472952+0000", 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.472953+0000", 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2899999999999999e-06 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.472974+0000", 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0729000000000001e-05 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.472996+0000", 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1878999999999998e-05 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.473012+0000", 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5916000000000001e-05 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.573196+0000", 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10018392800000001 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.573210+0000", 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3906999999999999e-05 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.574173+0000", 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00096345000000000003 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.574241+0000", 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8150000000000003e-05 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.581 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 42/31 ECSubWrite(tid=89, reqid=client.4568.0:279, at_version=42'41, trim_to=0'0, roll_forward_to=42'39))", 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:07.094046+0000", 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.432407574999999, 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103753032, 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.094046+0000", 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.094046+0000", 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.094048+0000", 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9630000000000001e-06 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.095150+0000", 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011015490000000001 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.095191+0000", 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1254999999999997e-05 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.195288+0000", 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100096488 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.195417+0000", 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012939399999999999 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.195447+0000", 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9889000000000001e-05 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.197703+0000", 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022562319999999999 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.197799+0000", 2024-04-18T23:11:17.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6261999999999994e-05 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 43/31 ECSubWrite(tid=91, reqid=client.4568.0:285, at_version=43'43, trim_to=0'0, roll_forward_to=42'41))", 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:07.691346+0000", 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.835108323, 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10290337400000001, 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691346+0000", 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691346+0000", 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691346+0000", 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691450+0000", 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103819 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691478+0000", 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7742e-05 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691495+0000", 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7102e-05 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.791721+0000", 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10022610699999999 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.791749+0000", 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7948000000000001e-05 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.794166+0000", 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024171309999999999 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.794249+0000", 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3139000000000005e-05 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 47/31 ECSubWrite(tid=116, reqid=client.4568.0:335, at_version=47'35, trim_to=0'0, roll_forward_to=47'33))", 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.936331+0000", 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.5901226560000001, 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10251489, 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.590 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936331+0000", 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936331+0000", 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936333+0000", 2024-04-18T23:11:17.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0729999999999999e-06 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936461+0000", 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012804099999999999 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936493+0000", 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1216000000000001e-05 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036630+0000", 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10013704399999999 2024-04-18T23:11:17.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036792+0000", 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016205599999999999 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036823+0000", 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1636e-05 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038757+0000", 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001933571 2024-04-18T23:11:17.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038846+0000", 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9252999999999994e-05 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 47/31 ECSubWrite(tid=118, reqid=client.4568.0:348, at_version=47'57, trim_to=0'0, roll_forward_to=44'55))", 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:13.165615+0000", 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.3608393699999999, 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10286374, 2024-04-18T23:11:17.594 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165615+0000", 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165615+0000", 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165616+0000", 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.753e-06 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165941+0000", 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000324323 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165970+0000", 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9572000000000001e-05 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.266070+0000", 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010000300000001 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.266260+0000", 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019000500000000001 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.266277+0000", 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6957e-05 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.268410+0000", 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021329729999999998 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.268478+0000", 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8153999999999998e-05 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.6s6 47/31 ECSubWrite(tid=20, reqid=client.4568.0:346, at_version=47'14, trim_to=0'0, roll_forward_to=43'12))", 2024-04-18T23:11:17.598 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:13.300001+0000", 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.2264533589999997, 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102818785, 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300001+0000", 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300001+0000", 2024-04-18T23:11:17.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300001+0000", 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3700000000000001e-07 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300296+0000", 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029461 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300331+0000", 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5785999999999999e-05 2024-04-18T23:11:17.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.400492+0000", 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10016021999999999 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.400641+0000", 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149667 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.400661+0000", 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9848999999999999e-05 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.402742+0000", 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002081009 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.402819+0000", 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7206999999999994e-05 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.602 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 47/31 ECSubWrite(tid=121, reqid=client.4568.0:352, at_version=47'38, trim_to=0'0, roll_forward_to=47'36))", 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:13.371242+0000", 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1552115470000004, 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102859354, 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371242+0000", 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371242+0000", 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371244+0000", 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.35e-06 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371676+0000", 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043242600000000002 2024-04-18T23:11:17.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371712+0000", 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5321000000000003e-05 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371726+0000", 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4436999999999999e-05 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.471965+0000", 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10023947699999999 2024-04-18T23:11:17.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.471982+0000", 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6269e-05 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.474035+0000", 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020528 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.474102+0000", 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7274000000000007e-05 2024-04-18T23:11:17.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:17.607 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:17.607 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:17.608 INFO:tasks.rados.rados.0.smithi142.stdout:153: finishing write tid 5 to smithi14230322-43 2024-04-18T23:11:17.608 INFO:tasks.rados.rados.0.smithi142.stdout:153: oid 43 updating version 0 to 4 2024-04-18T23:11:17.608 INFO:tasks.rados.rados.0.smithi142.stdout:153: oid 43 updating version 4 to 6 2024-04-18T23:11:17.608 INFO:tasks.rados.rados.0.smithi142.stdout:153: oid 43 updating version 6 to 7 2024-04-18T23:11:17.609 INFO:tasks.rados.rados.0.smithi142.stdout:153: oid 43 version 7 is already newer than 5 2024-04-18T23:11:17.609 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 43 v 7 (ObjNum 72 snap 11 seq_num 72) dirty exists 2024-04-18T23:11:17.609 INFO:tasks.rados.rados.0.smithi142.stdout:153: left oid 43 (ObjNum 72 snap 11 seq_num 72) 2024-04-18T23:11:17.609 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:11:17.610 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:17.606+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:17.664 INFO:tasks.rados.rados.0.smithi142.stdout:153: done (2 left) 2024-04-18T23:11:17.664 INFO:tasks.rados.rados.0.smithi142.stdout:155: done (1 left) 2024-04-18T23:11:17.664 INFO:tasks.rados.rados.0.smithi142.stdout:156: snap_create 2024-04-18T23:11:17.709 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:17.708+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:17.711 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-15 --journal-path /var/lib/ceph/osd/ceph-15/journal --force --op remove --pgid 3.5s0 2024-04-18T23:11:17.738 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:17.738 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:17.738 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:17.738 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:17.753 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:11:17.787 INFO:tasks.rados.rados.0.smithi142.stdout:154: finishing write tid 5 to smithi14230322-10 2024-04-18T23:11:17.787 INFO:tasks.rados.rados.0.smithi142.stdout:154: oid 10 updating version 0 to 32 2024-04-18T23:11:17.788 INFO:tasks.rados.rados.0.smithi142.stdout:154: oid 10 version 32 is already newer than 31 2024-04-18T23:11:17.788 INFO:tasks.rados.rados.0.smithi142.stdout:154: oid 10 updating version 32 to 34 2024-04-18T23:11:17.788 INFO:tasks.rados.rados.0.smithi142.stdout:154: oid 10 version 34 is already newer than 33 2024-04-18T23:11:17.788 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 10 v 34 (ObjNum 73 snap 11 seq_num 73) dirty exists 2024-04-18T23:11:17.788 INFO:tasks.rados.rados.0.smithi142.stdout:154: left oid 10 (ObjNum 73 snap 11 seq_num 73) 2024-04-18T23:11:17.809 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:17.806+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:17.911 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:17.906+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:17.915 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:17.915 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:17.915 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:17.915 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:17.915 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:17.930 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:11:18.012 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:18.010+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:18.052 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:18.052 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:18.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.673526000000003, 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50897375199999995, 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1 2024-04-18T23:11:18.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377587+0000", 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3089999999999998e-06 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377601+0000", 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3862999999999999e-05 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377623+0000", 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1741000000000001e-05 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377644+0000", 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1024000000000001e-05 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.576350+0000", 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198706355 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.576365+0000", 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4748e-05 2024-04-18T23:11:18.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.682700+0000", 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106335283 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.683098+0000", 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039820900000000001 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.683616+0000", 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051731399999999995 2024-04-18T23:11:18.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.688337+0000", 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047211509999999998 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886392+0000", 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198055008 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886494+0000", 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102203 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886559+0000", 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4543999999999997e-05 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:11:18.059 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456851013000001, 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41468469299999999, 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 110 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:11:18.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594261+0000", 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594266+0000", 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9690000000000002e-06 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594278+0000", 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2048999999999999e-05 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594286+0000", 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2900000000000002e-06 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.804556+0000", 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.21026968800000001 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.804784+0000", 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022826000000000001 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.805113+0000", 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032918500000000001 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.908604+0000", 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103490558 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008799+0000", 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100195153 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008880+0000", 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1329999999999996e-05 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008945+0000", 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4400999999999995e-05 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456813242999999, 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41616193699999998, 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.065 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 111 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:18.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6300000000000004e-07 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594305+0000", 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594317+0000", 2024-04-18T23:11:18.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1786e-05 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594327+0000", 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7249999999999992e-06 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906203+0000", 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.31187656499999999 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906336+0000", 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013266800000000001 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906542+0000", 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020588900000000001 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009954+0000", 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10341181100000001 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010185+0000", 2024-04-18T23:11:18.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023115700000000001 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010324+0000", 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013910800000000001 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010460+0000", 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000135635 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.45677251, 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42114173300000002, 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 112 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594339+0000", 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7700000000000004e-07 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594484+0000", 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000144804 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594500+0000", 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5752000000000001e-05 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594511+0000", 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1039000000000001e-05 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907178+0000", 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.31266760199999999 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907320+0000", 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014133099999999999 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907545+0000", 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022546 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010139+0000", 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102593936 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015192+0000", 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050530590000000004 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015364+0000", 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017223600000000001 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015480+0000", 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011573700000000001 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456583713000001, 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42137598300000001, 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.076 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 113 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594528+0000", 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594533+0000", 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0130000000000001e-06 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594545+0000", 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2009e-05 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594553+0000", 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7389999999999999e-06 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009008+0000", 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41445484999999999 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009125+0000", 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117773 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009350+0000", 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022510400000000001 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015618+0000", 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062671189999999998 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015662+0000", 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4619000000000001e-05 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015798+0000", 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013600600000000001 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015903+0000", 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104941 2024-04-18T23:11:18.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456546393, 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62069844100000005, 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.082 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 114 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594565+0000", 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594570+0000", 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.186e-06 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594582+0000", 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1763000000000001e-05 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594590+0000", 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5290000000000003e-06 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010572+0000", 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.415982252 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010727+0000", 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015558400000000001 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010971+0000", 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000243339 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015929+0000", 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049583639999999998 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215108+0000", 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19917934400000001 2024-04-18T23:11:18.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215199+0000", 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0556000000000005e-05 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215263+0000", 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3874000000000005e-05 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456510066, 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62089894099999998, 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 115 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594746+0000", 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014446700000000001 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594761+0000", 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4792e-05 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594769+0000", 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6160000000000002e-06 2024-04-18T23:11:18.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013742+0000", 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.418972438 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013974+0000", 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023265699999999999 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.014328+0000", 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035345 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018069+0000", 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037411079999999999 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215346+0000", 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19727757900000001 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215435+0000", 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8245999999999995e-05 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215500+0000", 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4968000000000005e-05 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456326815000001, 2024-04-18T23:11:18.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.72133309899999998, 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 116 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594785+0000", 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594790+0000", 2024-04-18T23:11:18.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4229999999999996e-06 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594803+0000", 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2196e-05 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594810+0000", 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6190000000000002e-06 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016062+0000", 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42125235 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016215+0000", 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000152051 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016420+0000", 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000205916 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018082+0000", 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001661718 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.315968+0000", 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29788616600000001 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316051+0000", 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2897000000000003e-05 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316117+0000", 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5866999999999998e-05 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.456289181999999, 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.721588585, 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.099 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 117 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:18.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594829+0000", 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7830000000000001e-06 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594841+0000", 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1505000000000001e-05 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.694940+0000", 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100099585 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016967+0000", 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32202692799999999 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017224+0000", 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000256548 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017583+0000", 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000359689 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.020139+0000", 2024-04-18T23:11:18.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025556649999999999 2024-04-18T23:11:18.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316256+0000", 2024-04-18T23:11:18.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29611688800000002 2024-04-18T23:11:18.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316345+0000", 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9285999999999999e-05 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316410+0000", 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5058e-05 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.106 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.976487793, 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.410819936, 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 140 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074624+0000", 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074804+0000", 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018036800000000001 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074824+0000", 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0018e-05 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074839+0000", 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4846e-05 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080014+0000", 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051746780000000003 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080068+0000", 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4345000000000002e-05 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080162+0000", 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4157e-05 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081144+0000", 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098186100000000011 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485220+0000", 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40407592199999998 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485314+0000", 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4161999999999997e-05 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485443+0000", 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012892900000000001 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.976254229999999, 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41085222500000002, 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.112 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 141 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:11:18.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074858+0000", 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0139999999999999e-06 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074863+0000", 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3759999999999999e-06 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074872+0000", 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3929999999999997e-06 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074883+0000", 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0722e-05 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080730+0000", 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058473249999999996 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080788+0000", 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7423999999999998e-05 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080881+0000", 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2938999999999999e-05 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081458+0000", 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057760199999999998 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485586+0000", 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40412787700000002 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485656+0000", 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0034000000000001e-05 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485709+0000", 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2519000000000003e-05 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.917028685999998, 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50946670199999999, 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 162 2024-04-18T23:11:18.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134083+0000", 2024-04-18T23:11:18.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7000000000000003e-07 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134088+0000", 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2719999999999997e-06 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134097+0000", 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1400000000000006e-06 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134104+0000", 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140241+0000", 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0061371919999999996 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140319+0000", 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7768000000000005e-05 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140446+0000", 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126637 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243778+0000", 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103331863 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643391+0000", 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.39961375599999999 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643476+0000", 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4274e-05 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643549+0000", 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3071000000000006e-05 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:18.123 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.916995526000001, 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50963481200000005, 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 163 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:18.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134116+0000", 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134123+0000", 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4930000000000003e-06 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134131+0000", 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9490000000000002e-06 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134138+0000", 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8399999999999997e-06 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140848+0000", 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067093389999999999 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140969+0000", 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121787 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141164+0000", 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019482700000000001 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243782+0000", 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10261785900000001 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643626+0000", 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.399843858 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643699+0000", 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2657999999999995e-05 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643750+0000", 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1481999999999999e-05 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.916961435000001, 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50988570200000005, 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 164 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134150+0000", 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4000000000000001e-07 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134332+0000", 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018177199999999999 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134351+0000", 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9321999999999999e-05 2024-04-18T23:11:18.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134368+0000", 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6994999999999999e-05 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141740+0000", 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073716420000000003 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141823+0000", 2024-04-18T23:11:18.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3042999999999998e-05 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141951+0000", 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127976 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243787+0000", 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101836063 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643874+0000", 2024-04-18T23:11:18.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40008676199999998 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643964+0000", 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0362e-05 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.644035+0000", 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1124999999999994e-05 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.916702198999999, 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.54739116099999996, 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 165 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134410+0000", 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5770000000000001e-06 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134420+0000", 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0175e-05 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134439+0000", 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8125e-05 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134453+0000", 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4956e-05 2024-04-18T23:11:18.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242648+0000", 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.108194819 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242783+0000", 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013431599999999999 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242992+0000", 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020903 2024-04-18T23:11:18.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.244280+0000", 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012883669999999999 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681587+0000", 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.43730722300000002 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681676+0000", 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8776000000000005e-05 2024-04-18T23:11:18.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681800+0000", 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012379700000000001 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:11 3.2s0 3:42341748:::smithi14230322-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.532812149000002, 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50900182400000005, 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.140 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 11 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518300+0000", 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.823e-06 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518324+0000", 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3238999999999999e-05 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518345+0000", 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0828000000000001e-05 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518359+0000", 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4912000000000001e-05 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521623+0000", 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003263808 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.522045+0000", 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00042140500000000002 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.523231+0000", 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011861840000000001 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530352+0000", 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071208340000000004 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.027150+0000", 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49679807100000001 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.027300+0000", 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015071999999999999 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:25 3.2s0 3:4250b374:::smithi14230322-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:18.145 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.529232249, 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.51011498, 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 25 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:18.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521881+0000", 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4430000000000002e-06 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524055+0000", 2024-04-18T23:11:18.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021740499999999999 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524086+0000", 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1414000000000001e-05 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524107+0000", 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0716e-05 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524323+0000", 2024-04-18T23:11:18.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215675 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524696+0000", 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037315500000000001 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526737+0000", 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020405380000000002 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:18.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530355+0000", 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036182720000000001 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.031805+0000", 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50145000299999998 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.031994+0000", 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018871400000000001 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:26 3.2s0 3:4250b374:::smithi14230322-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.526903155999999, 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60852922899999995, 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 26 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.151 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524210+0000", 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-04-18T23:11:18.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524220+0000", 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0056e-05 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524242+0000", 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2212e-05 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525734+0000", 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001491783 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528099+0000", 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023654380000000001 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528318+0000", 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000219065 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.529513+0000", 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011952530000000001 2024-04-18T23:11:18.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636414+0000", 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106900906 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132457+0000", 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49604301899999997 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132737+0000", 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027950099999999998 2024-04-18T23:11:18.155 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:31 3.2s0 3:42c34e84:::smithi14230322-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.524788732000001, 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60793644899999999, 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:18.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 31 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526324+0000", 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9400000000000001e-06 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526836+0000", 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051192200000000005 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526869+0000", 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2978000000000002e-05 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526890+0000", 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481999999999999e-05 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531167+0000", 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042762490000000002 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531508+0000", 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034109400000000001 2024-04-18T23:11:18.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.533620+0000", 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021122279999999998 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636418+0000", 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102798509 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134075+0000", 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.497656453 2024-04-18T23:11:18.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134259+0000", 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000183594 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:32 3.2s0 3:42c34e84:::smithi14230322-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.524185822, 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60874687999999999, 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:18.161 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 32 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:18.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526927+0000", 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0389999999999999e-06 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526936+0000", 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2599999999999994e-06 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:11:18.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0302e-05 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526973+0000", 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6359e-05 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634473+0000", 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.107499583 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634639+0000", 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016687200000000001 2024-04-18T23:11:18.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.635580+0000", 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094034999999999995 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636830+0000", 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012506220000000001 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135509+0000", 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49867835399999999 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:18.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135672+0000", 2024-04-18T23:11:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000163139 2024-04-18T23:11:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:18.166 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:18.166 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:18.168 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:11:18.168 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:18.168+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:18.268 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:18.266+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:18.299 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:18.299 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:18.299 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:11:18.299 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:18.313 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:11:18.369 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:18.364+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:18.434 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:18.434 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:18.435 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:18.435 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:18.435 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:18.449 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:11:18.470 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:18.468+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:18.496 INFO:tasks.rados.rados.0.smithi142.stdout:154: done (1 left) 2024-04-18T23:11:18.496 INFO:tasks.rados.rados.0.smithi142.stdout:156: done (0 left) 2024-04-18T23:11:18.496 INFO:tasks.rados.rados.0.smithi142.stdout:157: read oid 42 snap -1 2024-04-18T23:11:18.496 INFO:tasks.rados.rados.0.smithi142.stdout:157: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-18T23:11:18.496 INFO:tasks.rados.rados.0.smithi142.stdout:158: rmattr oid 17 current snap is 13 2024-04-18T23:11:18.496 INFO:tasks.rados.rados.0.smithi142.stdout:158: done (1 left) 2024-04-18T23:11:18.496 INFO:tasks.rados.rados.0.smithi142.stdout:159: setattr oid 28 current snap is 13 2024-04-18T23:11:18.499 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-04-18T23:11:18.503 INFO:tasks.rados.rados.0.smithi142.stdout:160: copy_from oid 33 from oid 16 current snap is 13 2024-04-18T23:11:18.503 INFO:tasks.rados.rados.0.smithi142.stdout:161: setattr oid 32 current snap is 13 2024-04-18T23:11:18.508 INFO:tasks.rados.rados.0.smithi142.stdout:162: snap_remove snap 2 2024-04-18T23:11:18.549 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:18.570 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:18.570+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:18.577 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:18.577 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:18.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:18.577 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:1 1.0 e23/22 1:30306672:devicehealth::main.db.0000000000000000:head v 23'1, mlcod=0'0)", 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:42.687401+0000", 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 35.891333342999999, 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198358914, 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687401+0000", 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687401+0000", 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687402+0000", 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687414+0000", 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2843000000000001e-05 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687440+0000", 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5398e-05 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.687463+0000", 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2901e-05 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.882317+0000", 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19485413500000001 2024-04-18T23:11:18.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.882396+0000", 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8681000000000001e-05 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.885671+0000", 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032758829999999998 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:42.885760+0000", 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8113000000000002e-05 2024-04-18T23:11:18.581 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:114 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'76, mlcod=28'72)", 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.011152+0000", 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.567582253000001, 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20343882299999999, 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011152+0000", 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011152+0000", 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011154+0000", 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.013e-06 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011169+0000", 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5665999999999999e-05 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011194+0000", 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4170999999999999e-05 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011209+0000", 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5766000000000001e-05 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.111484+0000", 2024-04-18T23:11:18.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100274822 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.111517+0000", 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2700999999999997e-05 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214524+0000", 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103007425 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214591+0000", 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6259000000000004e-05 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:115 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'77, mlcod=28'72)", 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.014595+0000", 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.564138711999998, 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20003684299999999, 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.586 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014595+0000", 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014595+0000", 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014596+0000", 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.153e-06 2024-04-18T23:11:18.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014657+0000", 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0216000000000001e-05 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014676+0000", 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9562e-05 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.014692+0000", 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5432999999999999e-05 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.113194+0000", 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098502135000000005 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.113228+0000", 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3834999999999997e-05 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214596+0000", 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101368415 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214632+0000", 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6093999999999997e-05 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:116 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'78, mlcod=28'74)", 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.016683+0000", 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.562051028999999, 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19884875900000001, 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.590 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016683+0000", 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016683+0000", 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016685+0000", 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8330000000000001e-06 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016696+0000", 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1396999999999999e-05 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016725+0000", 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8977999999999999e-05 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.016748+0000", 2024-04-18T23:11:18.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2728e-05 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214097+0000", 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19734953299999999 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214130+0000", 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2988000000000003e-05 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.215452+0000", 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013212670000000001 2024-04-18T23:11:18.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.215532+0000", 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0035000000000002e-05 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:117 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'79, mlcod=28'74)", 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.017827+0000", 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.560907308000001, 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20024583600000001, 2024-04-18T23:11:18.594 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017827+0000", 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017827+0000", 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017829+0000", 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8199999999999999e-06 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017840+0000", 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1392e-05 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017868+0000", 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7789000000000001e-05 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.017887+0000", 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8868999999999998e-05 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214695+0000", 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19680872499999999 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214729+0000", 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4060999999999998e-05 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.217987+0000", 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0032573459999999999 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.218073+0000", 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5834000000000005e-05 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:123 1.0 e28/22 1:30306672:devicehealth::main.db.0000000000000000:head v 28'85, mlcod=28'83)", 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.728930+0000", 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.849804408000001, 2024-04-18T23:11:18.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201634012, 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728930+0000", 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728930+0000", 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728931+0000", 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.316e-06 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728939+0000", 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3659999999999992e-06 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.728967+0000", 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7818999999999999e-05 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.829109+0000", 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100142192 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.929412+0000", 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10030241300000001 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.929448+0000", 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6331000000000002e-05 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.930477+0000", 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001028927 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.930564+0000", 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6648000000000001e-05 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:133 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'91, mlcod=28'85)", 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.301312247999999, 2024-04-18T23:11:18.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20553276400000001, 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277424+0000", 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.21e-06 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277436+0000", 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1693e-05 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277464+0000", 2024-04-18T23:11:18.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8608000000000001e-05 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377595+0000", 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100130684 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377730+0000", 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013455499999999999 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377781+0000", 2024-04-18T23:11:18.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1406999999999998e-05 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482870+0000", 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105089222 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482955+0000", 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4384999999999999e-05 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:141 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'99, mlcod=28'85)", 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.378148+0000", 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.200585929999999, 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107084213, 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378148+0000", 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378148+0000", 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378149+0000", 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8400000000000005e-07 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378153+0000", 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7029999999999997e-06 2024-04-18T23:11:18.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378164+0000", 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0276e-05 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.378170+0000", 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2659999999999998e-06 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.483345+0000", 2024-04-18T23:11:18.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105174774 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.483380+0000", 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5298e-05 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.485174+0000", 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017937420000000001 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.485232+0000", 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8669999999999999e-05 2024-04-18T23:11:18.610 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:148 1.0 e28/22 1:30306672:devicehealth::main.db.0000000000000000:head v 28'106, mlcod=28'104)", 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.825319+0000", 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.753414759000002, 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202174931, 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825319+0000", 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825319+0000", 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825326+0000", 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2439999999999996e-06 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825372+0000", 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5383000000000002e-05 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.825403+0000", 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0667999999999998e-05 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.925555+0000", 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100152493 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.025857+0000", 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100301604 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.025902+0000", 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5794000000000002e-05 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.027403+0000", 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015009769999999999 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.027494+0000", 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0767999999999996e-05 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:155 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'109, mlcod=28'105)", 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.443774137999998, 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20593233399999999, 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.615 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134962+0000", 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8560000000000001e-06 2024-04-18T23:11:18.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135074+0000", 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112357 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135099+0000", 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4705000000000001e-05 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135115+0000", 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5855999999999999e-05 2024-04-18T23:11:18.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135494+0000", 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037909800000000002 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135551+0000", 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7176999999999998e-05 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340852+0000", 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20530157900000001 2024-04-18T23:11:18.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340892+0000", 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9706000000000002e-05 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:156 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'110, mlcod=28'105)", 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.442967510999999, 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.205157487, 2024-04-18T23:11:18.619 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135768+0000", 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2130000000000001e-06 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135779+0000", 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1749000000000001e-05 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135794+0000", 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453e-05 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135807+0000", 2024-04-18T23:11:18.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2599000000000001e-05 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136859+0000", 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010521199999999999 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136911+0000", 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2096999999999999e-05 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340896+0000", 2024-04-18T23:11:18.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20398571500000001 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340924+0000", 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7540999999999999e-05 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:157 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'111, mlcod=28'105)", 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.442252819, 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20611499799999999, 2024-04-18T23:11:18.623 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136483+0000", 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.576e-06 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136499+0000", 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6529e-05 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136520+0000", 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0571999999999999e-05 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136532+0000", 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.223e-05 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237736+0000", 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10120357200000001 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237776+0000", 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0397e-05 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342561+0000", 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104784615 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342596+0000", 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5506999999999997e-05 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:164 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'118, mlcod=28'105)", 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.237791+0000", 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.340943033999999, 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10769007799999999, 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.627 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237791+0000", 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237791+0000", 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237791+0000", 2024-04-18T23:11:18.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4700000000000002e-07 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237871+0000", 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9401999999999996e-05 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237880+0000", 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8459999999999997e-06 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237886+0000", 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2230000000000004e-06 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.343134+0000", 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105248278 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.343183+0000", 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8810999999999997e-05 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.345405+0000", 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0022219010000000001 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.345481+0000", 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.627e-05 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:199 3.es0 3:78aee268:::smithi14230322-13:2 [read 0~212992,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.561530+0000", 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.017203525999999, 2024-04-18T23:11:18.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107316594, 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 199 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561530+0000", 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561530+0000", 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561534+0000", 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1930000000000002e-06 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561548+0000", 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4386000000000001e-05 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561583+0000", 2024-04-18T23:11:18.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4524999999999999e-05 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561600+0000", 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7419e-05 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561798+0000", 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019827500000000001 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561820+0000", 2024-04-18T23:11:18.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1934999999999999e-05 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.565718+0000", 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003897677 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.565935+0000", 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021744000000000001 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.668847+0000", 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102911744 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.635 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:200 3.es0 3:78aee268:::smithi14230322-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.561642+0000", 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.017092397999999, 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107866004, 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 200 2024-04-18T23:11:18.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561642+0000", 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561642+0000", 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561642+0000", 2024-04-18T23:11:18.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8000000000000004e-07 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561648+0000", 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.823e-06 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561677+0000", 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8535e-05 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561687+0000", 2024-04-18T23:11:18.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9569999999999997e-06 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562020+0000", 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000332985 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562037+0000", 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6892999999999999e-05 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.566800+0000", 2024-04-18T23:11:18.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0047631749999999997 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.567077+0000", 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000277447 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.669508+0000", 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102430309 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.640 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:201 3.es0 3:78aee268:::smithi14230322-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.016978399999999, 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41057925299999998, 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:11:18.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561757+0000", 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.47e-07 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561762+0000", 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.04e-06 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561777+0000", 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4962e-05 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561785+0000", 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4500000000000004e-06 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562118+0000", 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000332882 2024-04-18T23:11:18.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562139+0000", 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1101999999999999e-05 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.769814+0000", 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.207675318 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770164+0000", 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034956199999999998 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.972335+0000", 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20217098999999999 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.4.21:1 3.es0 3:7a4badf6:::smithi14230322-17:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.817201567000001, 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31203379199999998, 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761535+0000", 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7999999999999999e-06 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761547+0000", 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1279e-05 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761568+0000", 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1549e-05 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761588+0000", 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0426000000000001e-05 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770853+0000", 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092644660000000007 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.771105+0000", 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025160800000000001 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073566+0000", 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30246166400000002 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:333 3.es0 3:799dcc49:::smithi14230322-39:head [create,setxattr Yjk]k``VRNVkfm]Rnjieeerecjk`Wm (15648) in=30b,setxattr QgpqijWTrohYWOsYikii[YdsSee`QrtrZ]nSPTUt (1225) in=40b,setxattr l]iRdbVhahjiOeVf`Yc\\U_OYjY]nr[js\\NggsVVY (1015) in=40b,setxattr b^W[ZgXWiRmfON^`rihabiO]lNkro`P (15648) in=31b,setxattr VReWeRiYgNSZU\\d\\YOrNS`]ogtdOjVijWVeY`fhd (1051) in=40b,setxattr SYsfepqPdsc`cUg\\WaTfgrcUYhUbTiaQY_`rlt (15648) in=38b,setxattr P[`jheVP[fjpnXqdjPhddi_oWUf\\dWWbSlodc`eS (1086) in=40b,setxattr X[^VkT`QVNea_jfmlVkdafY]hrorgVhl]fqtiVie (1226) in=40b,setxattr Xs\\WjfX^_Ss_RPr_tT^a]fnSSlYrcPTbmSaksi_c (19437) in=40b,setxattr QUn_^p]__nRhn\\srj]jla^iQSPOpgQcVcQZU^ool (1823) in=40b,setxattr f\\^WXdjheR\\aQSYk]Un_^hkrZTSVdhiZqgcin[Sa (1155) in=40b,setxattr bkb]mNeZddhQrcsORPSgqbWecOTtcPYZaRQOg`nY (1614) in=40b,setxattr _T_atc\\\\Rml^aOPkNsbsb^VatOWSs\\URSjVqfe\\Z (1226) in=40b,setxattr obYqlk]hgNOQgXa^ZhalmPcXl\\lkOdNNiYlOfdhq (1612) in=40b,setxattr VsbnoacTrXVpb]PV]kVPYbQP_qefUdfOgk`qUaNa (1999) in=40b,setxattr ZqTlaiiPajkNT^ce\\W\\oklalkiT`\\\\nhjcgpZrUa (3297) in=40b,setxattr QfUmVl\\][PRg\\ajWQS`]Oe^Zlig]b]Rs_o]NsY[i (244) in=40b,setxattr R_Ucr`a\\j]Ri_YSp[gamgRSRd`]UYnPRspctZ`R] (4068) in=40b,setxattr aj`m[hpi__Rrg_Z[m^rVOVrVgk^cS_]piWV]lkRZ (947) in=40b,setxattr ^nWmd_TeciTqUOnWXn_jXch]heNac[WdcqdPbiT^ (1647) in=40b] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9513782050000001, 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30962595399999998, 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627359+0000", 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3400000000000002e-06 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627676+0000", 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000317144 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627710+0000", 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3855e-05 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627753+0000", 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3341999999999999e-05 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627926+0000", 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017248799999999999 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627946+0000", 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9868999999999999e-05 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729527+0000", 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101581587 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729747+0000", 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021955299999999999 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.733259+0000", 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035115820000000001 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739147+0000", 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058886169999999996 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936726+0000", 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.197578422 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936982+0000", 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025615499999999999 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.654 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:344 3.es0 3:7a4badf6:::smithi14230322-17:head [delete] snapc 8=[7,4] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.9373196479999999, 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.295980827, 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 344 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641417+0000", 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2630000000000002e-06 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641427+0000", 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0674e-05 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641460+0000", 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2768999999999997e-05 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641482+0000", 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1959000000000001e-05 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641642+0000", 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016009100000000001 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641653+0000", 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.128e-05 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735170+0000", 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.093517063999999997 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735466+0000", 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029521400000000002 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.736750+0000", 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012846540000000001 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739533+0000", 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027831589999999999 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937242+0000", 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19770827199999999 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937395+0000", 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000153428 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.fs1 51/31 ECSubWrite(tid=24, reqid=client.4568.0:385, at_version=51'5, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:18.660 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:16.874679+0000", 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.7040551260000001, 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20129744899999999, 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.874679+0000", 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.874679+0000", 2024-04-18T23:11:18.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.874680+0000", 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5099999999999999e-06 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.875037+0000", 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035653300000000002 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.875067+0000", 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9759000000000001e-05 2024-04-18T23:11:18.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.975213+0000", 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.1001464 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.975373+0000", 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000159932 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.975392+0000", 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8626000000000001e-05 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:17.075916+0000", 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100524554 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:17.075976+0000", 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0134999999999999e-05 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:18.664 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:18.666 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:11:18.671 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:18.666+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:18.718 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:18.718 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:18.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:18.739 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:18.757 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:18.757 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:18.772 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:18.768+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:18.778 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:18.778 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:18.794 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:18.794 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:18.818 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:18.818 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:18.837 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:18.838 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:18.838 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:18.838 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:18.853 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:11:18.870 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:18.873 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:18.872+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:18.892 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:18.892 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:18.915 INFO:teuthology.orchestra.run.smithi142.stdout: marking collection for removal 2024-04-18T23:11:18.915 INFO:teuthology.orchestra.run.smithi142.stdout:setting '_remove' omap key 2024-04-18T23:11:18.915 INFO:teuthology.orchestra.run.smithi142.stdout:finish_remove_pgs 3.5s0_head removing 3.5s0 2024-04-18T23:11:18.916 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:18.916 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:18.922 INFO:teuthology.orchestra.run.smithi142.stdout:Remove successful 2024-04-18T23:11:18.936 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:18.936 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:18.957 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:18.957 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:18.973 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:18.970+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:18.976 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:18.976 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:18.977 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:18.977 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:18.990 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:11:18.999 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:18.999 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:19.017 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:19.017 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:19.018 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENXIO: problem getting command descriptions from osd.15 2024-04-18T23:11:19.024 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T23:11:19.025 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T23:11:19.074 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:19.070+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:19.112 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:19.113 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:19.128 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:11:19.175 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:19.172+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:19.276 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:19.274+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:19.293 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:19.293 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:19.309 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:11:19.376 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:19.374+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:19.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-15 --journal-path /var/lib/ceph/osd/ceph-15/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.3.5s0.15 2024-04-18T23:11:19.476 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:19.476 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:19.490 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:11:19.498 INFO:tasks.rados.rados.0.smithi142.stdout:162: done (4 left) 2024-04-18T23:11:19.498 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 25 current snap is 13 2024-04-18T23:11:19.498 INFO:tasks.rados.rados.0.smithi142.stdout:163: seq_num 74 ranges {770048=475136,1245184=622592,1867776=147456} 2024-04-18T23:11:19.498 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:19.496+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:19.518 INFO:tasks.rados.rados.0.smithi142.stdout:163: writing smithi14230322-25 from 770048 to 1245184 tid 1 2024-04-18T23:11:19.528 INFO:tasks.rados.rados.0.smithi142.stdout:163: writing smithi14230322-25 from 1245184 to 1867776 tid 2 2024-04-18T23:11:19.531 INFO:tasks.rados.rados.0.smithi142.stdout:163: writing smithi14230322-25 from 1867776 to 2015232 tid 3 2024-04-18T23:11:19.531 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 35 current snap is 13 2024-04-18T23:11:19.531 INFO:tasks.rados.rados.0.smithi142.stdout:164: seq_num 75 ranges {1720320=491520,2211840=770048} 2024-04-18T23:11:19.557 INFO:tasks.rados.rados.0.smithi142.stdout:164: writing smithi14230322-35 from 1720320 to 2211840 tid 1 2024-04-18T23:11:19.563 INFO:tasks.rados.rados.0.smithi142.stdout:164: writing smithi14230322-35 from 2211840 to 2981888 tid 2 2024-04-18T23:11:19.563 INFO:tasks.rados.rados.0.smithi142.stdout:165: snap_remove snap 12 2024-04-18T23:11:19.578 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:19.576+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:19.613 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:19.613 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:19.627 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:11:19.679 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:19.676+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:19.756 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:19.756 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:19.770 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:11:19.780 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:19.778+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:19.881 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:19.880+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:19.890 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:19.891 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:19.905 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:11:19.982 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:19.978+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:20.028 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:20.028 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:20.043 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:11:20.083 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:20.084+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:20.169 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:20.169 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:20.184 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:20.186+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:20.185 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:11:20.285 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:20.280+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:20.300 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:20.300 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:20.314 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:11:20.385 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:20.382+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:20.470 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:20.471 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:20.485 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:11:20.486 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:20.488+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:20.504 INFO:tasks.rados.rados.0.smithi142.stdout:165: done (6 left) 2024-04-18T23:11:20.504 INFO:tasks.rados.rados.0.smithi142.stdout:166: delete oid 31 current snap is 13 2024-04-18T23:11:20.504 INFO:tasks.rados.rados.0.smithi142.stdout:160: finishing copy_from racing read to smithi14230322-33 2024-04-18T23:11:20.504 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 28 v 18 (ObjNum 33 snap 0 seq_num 3924760864) dirty exists 2024-04-18T23:11:20.504 INFO:tasks.rados.rados.0.smithi142.stdout:157: expect (ObjNum 41 snap 0 seq_num 41) 2024-04-18T23:11:20.524 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 32 v 61 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:160: finishing copy_from to smithi14230322-33 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 33 v 8 (ObjNum 57 snap 4 seq_num 57) dirty exists 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:164: finishing write tid 1 to smithi14230322-35 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:164: finishing write tid 2 to smithi14230322-35 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:164: finishing write tid 3 to smithi14230322-35 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:163: finishing write tid 1 to smithi14230322-25 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:164: finishing write tid 4 to smithi14230322-35 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:164: oid 35 updating version 0 to 46 2024-04-18T23:11:20.528 INFO:tasks.rados.rados.0.smithi142.stdout:164: oid 35 updating version 46 to 47 2024-04-18T23:11:20.529 INFO:tasks.rados.rados.0.smithi142.stdout:164: oid 35 updating version 47 to 48 2024-04-18T23:11:20.529 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 35 v 48 (ObjNum 75 snap 13 seq_num 75) dirty exists 2024-04-18T23:11:20.529 INFO:tasks.rados.rados.0.smithi142.stdout:164: left oid 35 (ObjNum 75 snap 13 seq_num 75) 2024-04-18T23:11:20.529 INFO:tasks.rados.rados.0.smithi142.stdout:163: finishing write tid 2 to smithi14230322-25 2024-04-18T23:11:20.529 INFO:tasks.rados.rados.0.smithi142.stdout:163: finishing write tid 3 to smithi14230322-25 2024-04-18T23:11:20.538 INFO:tasks.rados.rados.0.smithi142.stdout:163: finishing write tid 4 to smithi14230322-25 2024-04-18T23:11:20.587 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:20.582+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:20.599 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:20.599 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:20.611 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:11:20.665 INFO:teuthology.orchestra.run.smithi142.stdout:get_pg_num_history pg_num_history pg_num_history(e48 pg_nums {1={22=1},2={25=8},3={31=16}} deleted_pools ) 2024-04-18T23:11:20.665 INFO:teuthology.orchestra.run.smithi142.stdout:Importing pgid 3.5s0 2024-04-18T23:11:20.666 INFO:teuthology.orchestra.run.smithi142.stdout:Write 0#3:a09a6894:::smithi14230322-25:6# 2024-04-18T23:11:20.666 INFO:teuthology.orchestra.run.smithi142.stdout:Write 0#3:a09a6894:::smithi14230322-25:head# 2024-04-18T23:11:20.667 INFO:teuthology.orchestra.run.smithi142.stdout:snapset 6=[]:{6=[4]} 2024-04-18T23:11:20.668 INFO:teuthology.orchestra.run.smithi142.stdout:Write 0#3:a6fe874e:::smithi14230322-33:5# 2024-04-18T23:11:20.668 INFO:teuthology.orchestra.run.smithi142.stdout:Write 0#3:a6fe874e:::smithi14230322-33:head# 2024-04-18T23:11:20.668 INFO:teuthology.orchestra.run.smithi142.stdout:snapset 5=[]:{5=[4]} 2024-04-18T23:11:20.669 INFO:teuthology.orchestra.run.smithi142.stdout:Write 0#3:a832c25e:::smithi14230322-46:head# 2024-04-18T23:11:20.669 INFO:teuthology.orchestra.run.smithi142.stdout:snapset 0=[]:{} 2024-04-18T23:11:20.669 INFO:teuthology.orchestra.run.smithi142.stdout:write_pg epoch 48 info 3.5s0( v 44'25 (0'0,44'25] local-lis/les=31/32 n=5 ec=31/31 lis/c=31/31 les/c/f=32/32/0 sis=31) 2024-04-18T23:11:20.671 INFO:teuthology.orchestra.run.smithi142.stdout:Import successful 2024-04-18T23:11:20.688 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:20.686+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:20.743 INFO:tasks.rados.rados.0.smithi142.stdout:163: finishing write tid 5 to smithi14230322-25 2024-04-18T23:11:20.743 INFO:tasks.rados.rados.0.smithi142.stdout:163: oid 25 updating version 0 to 33 2024-04-18T23:11:20.744 INFO:tasks.rados.rados.0.smithi142.stdout:163: oid 25 updating version 33 to 34 2024-04-18T23:11:20.744 INFO:tasks.rados.rados.0.smithi142.stdout:163: oid 25 version 34 is already newer than 32 2024-04-18T23:11:20.744 INFO:tasks.rados.rados.0.smithi142.stdout:163: oid 25 version 34 is already newer than 31 2024-04-18T23:11:20.744 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 25 v 34 (ObjNum 74 snap 13 seq_num 74) dirty exists 2024-04-18T23:11:20.744 INFO:tasks.rados.rados.0.smithi142.stdout:163: left oid 25 (ObjNum 74 snap 13 seq_num 74) 2024-04-18T23:11:20.767 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:20.767 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:20.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:11:20.789 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:20.788+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:20.890 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:20.888+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:20.940 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:20.940 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:20.955 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:11:20.991 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:20.990+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:21.010 INFO:tasks.rados.rados.0.smithi142.stdout:157: done (6 left) 2024-04-18T23:11:21.010 INFO:tasks.rados.rados.0.smithi142.stdout:159: done (5 left) 2024-04-18T23:11:21.010 INFO:tasks.rados.rados.0.smithi142.stdout:160: done (4 left) 2024-04-18T23:11:21.010 INFO:tasks.rados.rados.0.smithi142.stdout:161: done (3 left) 2024-04-18T23:11:21.010 INFO:tasks.rados.rados.0.smithi142.stdout:163: done (2 left) 2024-04-18T23:11:21.010 INFO:tasks.rados.rados.0.smithi142.stdout:164: done (1 left) 2024-04-18T23:11:21.010 INFO:tasks.rados.rados.0.smithi142.stdout:166: done (0 left) 2024-04-18T23:11:21.010 INFO:tasks.rados.rados.0.smithi142.stdout:167: snap_create 2024-04-18T23:11:21.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:21.091 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:21.090+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:21.120 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:21.120 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:21.134 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:11:21.183 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:21.183 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:21.192 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:21.192+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:21.204 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:21.204 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:21.212 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f /var/log/ceph/exp.3.5s0.15 2024-04-18T23:11:21.226 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:21.227 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:21.247 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:21.248 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:21.264 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:21.264 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:21.285 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:21.285 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:21.292 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:21.290+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:21.302 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:21.307 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:21.307 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:21.316 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:11:21.326 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:21.326 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:21.347 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:21.347 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:21.368 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:21.368 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:21.389 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:21.389 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:21.402 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:21.400+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:21.407 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:21.407 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:21.424 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:21.445 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:21.445 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:21.466 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENXIO: problem getting command descriptions from osd.15 2024-04-18T23:11:21.470 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T23:11:21.471 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T23:11:21.482 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.1s1 33/31 ECSubWrite(tid=3, reqid=client.4568.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.527595+0000", 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 24.953398645, 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29580373399999998, 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.483 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527595+0000", 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527595+0000", 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527596+0000", 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1600000000000001e-07 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527623+0000", 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6698000000000001e-05 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527644+0000", 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1946e-05 2024-04-18T23:11:21.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527665+0000", 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0859000000000001e-05 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.722634+0000", 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19496872100000001 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.722649+0000", 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4832e-05 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.823340+0000", 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100690934 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.823399+0000", 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9228000000000003e-05 2024-04-18T23:11:21.487 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:192 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 20.920556176000002, 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30501581700000002, 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:21.488 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 192 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560439+0000", 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6029999999999999e-06 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560450+0000", 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0835999999999999e-05 2024-04-18T23:11:21.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560478+0000", 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7619000000000001e-05 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560489+0000", 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0998999999999999e-05 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560680+0000", 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019082800000000001 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560688+0000", 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2260000000000005e-06 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662105+0000", 2024-04-18T23:11:21.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10141672 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662220+0000", 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115872 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.865454+0000", 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20323311399999999 2024-04-18T23:11:21.493 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:193 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 20.920447430999999, 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30581751899999998, 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.494 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 193 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560548+0000", 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560554+0000", 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2230000000000004e-06 2024-04-18T23:11:21.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560566+0000", 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2622999999999999e-05 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560575+0000", 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1589999999999993e-06 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563654+0000", 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030794709999999999 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563665+0000", 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0708e-05 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662682+0000", 2024-04-18T23:11:21.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099017665000000005 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.663181+0000", 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049841999999999996 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.866364+0000", 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20318328699999999 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.499 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:209 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560597+0000", 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 20.920396750999998, 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.206586359, 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.500 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 209 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560597+0000", 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560597+0000", 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560598+0000", 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6000000000000003e-07 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560603+0000", 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.057e-06 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560617+0000", 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3801999999999999e-05 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560625+0000", 2024-04-18T23:11:21.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6869999999999994e-06 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560749+0000", 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012451399999999999 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560769+0000", 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0222000000000001e-05 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.504 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.564039+0000", 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003269831 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.564169+0000", 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000130147 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767184+0000", 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20301433899999999 2024-04-18T23:11:21.505 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:210 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 20.920349822999999, 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.41278097800000002, 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:21.506 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 210 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560645+0000", 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1829999999999999e-06 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560652+0000", 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3500000000000002e-06 2024-04-18T23:11:21.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560669+0000", 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6833e-05 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560682+0000", 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3585e-05 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.509 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560856+0000", 2024-04-18T23:11:21.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017400200000000001 2024-04-18T23:11:21.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:21.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560874+0000", 2024-04-18T23:11:21.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8386e-05 2024-04-18T23:11:21.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767285+0000", 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20641079700000001 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767423+0000", 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013740399999999999 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.973425+0000", 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20600243800000001 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:211 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 20.920261527000001, 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51049911299999995, 2024-04-18T23:11:21.512 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 211 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560734+0000", 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-06 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560741+0000", 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1559999999999998e-06 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560758+0000", 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7382999999999999e-05 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560775+0000", 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7051999999999999e-05 2024-04-18T23:11:21.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560940+0000", 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164659 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560949+0000", 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7630000000000006e-06 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767943+0000", 2024-04-18T23:11:21.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.206994276 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.768107+0000", 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164358 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.071232+0000", 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30312427600000003 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.517 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/31 ECSubWrite(tid=58, reqid=client.4568.0:196, at_version=36'21, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.761778+0000", 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 20.719216364000001, 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20846394800000001, 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761778+0000", 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761778+0000", 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761779+0000", 2024-04-18T23:11:21.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.646e-06 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761841+0000", 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1684000000000001e-05 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761871+0000", 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0139000000000001e-05 2024-04-18T23:11:21.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.761888+0000", 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6439e-05 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.765276+0000", 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033887309999999999 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:21.521 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.765291+0000", 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4576000000000001e-05 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970157+0000", 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20486607000000001 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970242+0000", 2024-04-18T23:11:21.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4662999999999999e-05 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 36/31 ECSubWrite(tid=59, reqid=client.4568.0:197, at_version=36'22, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.762534+0000", 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 20.718460097000001, 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.208314309, 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.523 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762534+0000", 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762534+0000", 2024-04-18T23:11:21.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762535+0000", 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.223e-06 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762562+0000", 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6974999999999998e-05 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762583+0000", 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1126000000000001e-05 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.762595+0000", 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2140000000000001e-05 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.766273+0000", 2024-04-18T23:11:21.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036778969999999999 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.766297+0000", 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3289e-05 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970782+0000", 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20448570899999999 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.970848+0000", 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5950000000000004e-05 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:359 3.cs0 3:35606aaf:::smithi14230322-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.2847455770000007, 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.36957785599999998, 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.528 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 359 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196249+0000", 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5700000000000002e-07 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196253+0000", 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8979999999999998e-06 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196271+0000", 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8270999999999998e-05 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196280+0000", 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.85e-06 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196650+0000", 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00037011599999999998 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196698+0000", 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8060999999999999e-05 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565575+0000", 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.36887705999999998 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565695+0000", 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119663 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565826+0000", 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013118 2024-04-18T23:11:21.533 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:384 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~442368] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.8164522730000003, 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51544569399999995, 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.534 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 384 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664544+0000", 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6240000000000002e-06 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665037+0000", 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049266399999999997 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665070+0000", 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2963000000000003e-05 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665089+0000", 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8672e-05 2024-04-18T23:11:21.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665233+0000", 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014389900000000001 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665245+0000", 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2446000000000001e-05 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769742+0000", 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104497225 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.770038+0000", 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029553099999999997 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.772939+0000", 2024-04-18T23:11:21.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029013009999999998 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781282+0000", 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0083427859999999996 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.179880+0000", 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.398598591 2024-04-18T23:11:21.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.179987+0000", 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010699199999999999 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:385 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~720896] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:21.541 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.8123786480000001, 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61243552499999998, 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 385 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:11:21.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:11:21.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668618+0000", 2024-04-18T23:11:21.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5129999999999999e-06 2024-04-18T23:11:21.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669576+0000", 2024-04-18T23:11:21.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000957726 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669605+0000", 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9566000000000001e-05 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669622+0000", 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6409000000000002e-05 2024-04-18T23:11:21.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.775317+0000", 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105695331 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.775501+0000", 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018403000000000001 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777261+0000", 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001759652 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781284+0000", 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040233049999999996 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:21.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.280844+0000", 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49956001900000002 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281051+0000", 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020697399999999999 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:386 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~49152] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.8113415780000004, 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61210873300000002, 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.549 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 386 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:11:21.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669654+0000", 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.297e-06 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669730+0000", 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6474e-05 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669748+0000", 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7598999999999999e-05 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769883+0000", 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100135671 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.778786+0000", 2024-04-18T23:11:21.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0089029639999999993 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.779085+0000", 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029830699999999999 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.780258+0000", 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011728070000000001 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783446+0000", 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031888179999999999 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281593+0000", 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49814646800000001 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281761+0000", 2024-04-18T23:11:21.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016832800000000001 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:387 3.fs0 3:f47c3018:::smithi14230322-43:head [setxattr _header (57) in=7b] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.7110710220000005, 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51243170900000001, 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.555 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 387 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769925+0000", 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7799999999999999e-06 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.557 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769934+0000", 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6460000000000005e-06 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769956+0000", 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1216000000000002e-05 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.558 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769967+0000", 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-05 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781530+0000", 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.01156212 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781688+0000", 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015813200000000001 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.782502+0000", 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00081454100000000001 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783689+0000", 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001186552 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.282098+0000", 2024-04-18T23:11:21.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49840956600000003 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.282355+0000", 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025625600000000001 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:388 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.7110135939999997, 2024-04-18T23:11:21.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81817408400000002, 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 388 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769981+0000", 2024-04-18T23:11:21.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6400000000000001e-07 2024-04-18T23:11:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769985+0000", 2024-04-18T23:11:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7689999999999998e-06 2024-04-18T23:11:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.564 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769996+0000", 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.075e-05 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.770003+0000", 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5769999999999996e-06 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783045+0000", 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013041308999999999 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783303+0000", 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025812900000000001 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.283544+0000", 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50024145900000005 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.283825+0000", 2024-04-18T23:11:21.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028086099999999998 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.588154+0000", 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30432946599999999 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 51/31 ECSubWriteReply(tid=26, last_complete=51'7, committed=1, applied=1))", 2024-04-18T23:11:21.567 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.979048+0000", 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.5019461710000002, 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.301182966, 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.979048+0000", 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.979048+0000", 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.979049+0000", 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6700000000000002e-07 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.569 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.979055+0000", 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.6359999999999997e-06 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.979073+0000", 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7598999999999999e-05 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.979082+0000", 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.884e-06 2024-04-18T23:11:21.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.280206+0000", 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30112381399999999 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.280231+0000", 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5066e-05 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.571 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 51/31 ECSubWriteReply(tid=25, last_complete=51'6, committed=1, applied=1))", 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.980275+0000", 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.5007191300000002, 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29988973099999999, 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980275+0000", 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980275+0000", 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980276+0000", 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7000000000000003e-07 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980282+0000", 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0000000000000002e-06 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980302+0000", 2024-04-18T23:11:21.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0156000000000001e-05 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980319+0000", 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6562000000000002e-05 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.280132+0000", 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29981376999999998 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.280165+0000", 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2273000000000001e-05 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 51/31 ECSubWriteReply(tid=26, last_complete=51'7, committed=1, applied=1))", 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.980919+0000", 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.5000745479999997, 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30028848299999999, 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.576 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980919+0000", 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980919+0000", 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980921+0000", 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4470000000000001e-06 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980931+0000", 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0132999999999999e-05 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980961+0000", 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9522999999999998e-05 2024-04-18T23:11:21.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980977+0000", 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6206e-05 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281172+0000", 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30019530900000002 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281208+0000", 2024-04-18T23:11:21.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5865000000000003e-05 2024-04-18T23:11:21.580 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.580 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.580 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.580 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.fs0 51/31 ECSubReadReply(tid=28, attrs_read=0))", 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.1959021270000001, 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30201136499999998, 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285093+0000", 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.08e-06 2024-04-18T23:11:21.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285111+0000", 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8272000000000001e-05 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285129+0000", 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7706999999999998e-05 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285140+0000", 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1472e-05 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587038+0000", 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30189753600000002 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587103+0000", 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5297999999999997e-05 2024-04-18T23:11:21.584 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.fs0 51/31 ECSubRead(tid=28, to_read={3:f47c3018:::smithi14230322-43:head=0,4096,0}, subchunks={3:f47c3018:::smithi14230322-43:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.1953019229999997, 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30165329400000002, 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.585 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285725+0000", 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3365999999999999e-05 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.385871+0000", 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100145397 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587204+0000", 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.201332972 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587345+0000", 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000141559 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.fs0 51/31 ECSubReadReply(tid=28, attrs_read=0))", 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:17.286253+0000", 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.1947406059999999, 2024-04-18T23:11:21.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.301234422, 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.286253+0000", 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.286253+0000", 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.286254+0000", 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.286269+0000", 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4517e-05 2024-04-18T23:11:21.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.286286+0000", 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7339000000000001e-05 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.286298+0000", 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1423000000000001e-05 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587420+0000", 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30112287599999998 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587488+0000", 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7304000000000005e-05 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:21.593 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:21.594 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-04-18T23:11:21.596 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:21.590+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:21.597 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:11:21.598 INFO:tasks.rados.rados.0.smithi142.stdout:167: done (0 left) 2024-04-18T23:11:21.598 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 31 current snap is 14 2024-04-18T23:11:21.598 INFO:tasks.rados.rados.0.smithi142.stdout:168: seq_num 77 ranges {0=524288,524288=524288,1048576=245760} 2024-04-18T23:11:21.598 INFO:tasks.rados.rados.0.smithi142.stdout:168: writing smithi14230322-31 from 0 to 524288 tid 1 2024-04-18T23:11:21.598 INFO:tasks.rados.rados.0.smithi142.stdout:168: writing smithi14230322-31 from 524288 to 1048576 tid 2 2024-04-18T23:11:21.598 INFO:tasks.rados.rados.0.smithi142.stdout:168: writing smithi14230322-31 from 1048576 to 1294336 tid 3 2024-04-18T23:11:21.598 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 16 current snap is 14 2024-04-18T23:11:21.599 INFO:tasks.rados.rados.0.smithi142.stdout:169: seq_num 78 ranges {1228800=557056,1785856=671744,2457600=98304} 2024-04-18T23:11:21.599 INFO:tasks.rados.rados.0.smithi142.stdout:169: writing smithi14230322-16 from 1228800 to 1785856 tid 1 2024-04-18T23:11:21.599 INFO:tasks.rados.rados.0.smithi142.stdout:169: writing smithi14230322-16 from 1785856 to 2457600 tid 2 2024-04-18T23:11:21.599 INFO:tasks.rados.rados.0.smithi142.stdout:169: writing smithi14230322-16 from 2457600 to 2555904 tid 3 2024-04-18T23:11:21.599 INFO:tasks.rados.rados.0.smithi142.stdout:170: copy_from oid 10 from oid 39 current snap is 14 2024-04-18T23:11:21.599 INFO:tasks.rados.rados.0.smithi142.stdout:171: read oid 48 snap 9 2024-04-18T23:11:21.599 INFO:tasks.rados.rados.0.smithi142.stdout:171: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:21.599 INFO:tasks.rados.rados.0.smithi142.stdout:172: copy_from oid 50 from oid 1 current snap is 14 2024-04-18T23:11:21.599 INFO:tasks.rados.rados.0.smithi142.stdout:173: snap_remove snap 8 2024-04-18T23:11:21.696 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:21.696+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:417 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~524288 in=524288b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.25705668100000001, 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.25712859900000001, 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.778 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 417 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:11:21.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520104+0000", 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2389999999999999e-06 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520670+0000", 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00056593599999999996 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520690+0000", 2024-04-18T23:11:21.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9945e-05 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520699+0000", 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0429999999999996e-06 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618217+0000", 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097518176999999998 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618227+0000", 2024-04-18T23:11:21.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7860000000000008e-06 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622364+0000", 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041371389999999997 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622653+0000", 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028858600000000002 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:21.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.624345+0000", 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001691782 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727165+0000", 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102820087 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:418 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~245760 in=245760b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.25557382499999998, 2024-04-18T23:11:21.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.25580593899999998, 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 418 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521587+0000", 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2100000000000001e-06 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521960+0000", 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000373402 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521972+0000", 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.207e-05 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521979+0000", 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7839999999999998e-06 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725251+0000", 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20327163200000001 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725428+0000", 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000177071 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.726423+0000", 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00099555200000000003 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.787 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:419 3.6s0 3.ffc08ac6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.25516880800000002, 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.25551038300000001, 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.789 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 419 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2200000000000004e-07 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521994+0000", 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8289999999999999e-06 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521999+0000", 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8470000000000003e-06 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522003+0000", 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1250000000000003e-06 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:420 3.6s0 3.ffc08ac6 (undecoded) ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.25515113499999997, 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.255574511, 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:21.792 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 420 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:11:21.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522009+0000", 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4999999999999998e-07 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522012+0000", 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.819e-06 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522017+0000", 2024-04-18T23:11:21.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2100000000000001e-06 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522021+0000", 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1400000000000002e-06 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=25, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.627081+0000", 2024-04-18T23:11:21.795 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.150078197, 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.15058418400000001, 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627081+0000", 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627081+0000", 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627083+0000", 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.443e-06 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627091+0000", 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0469999999999994e-06 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627114+0000", 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2679000000000001e-05 2024-04-18T23:11:21.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627125+0000", 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1513e-05 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=25, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.727297+0000", 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.049862584000000001, 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.050437759999999998, 2024-04-18T23:11:21.798 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727297+0000", 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727297+0000", 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727298+0000", 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.125e-06 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727303+0000", 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6619999999999996e-06 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727317+0000", 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3808999999999999e-05 2024-04-18T23:11:21.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727327+0000", 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0565e-05 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=26, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.728273+0000", 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.048886822000000003, 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.049530523, 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.801 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728273+0000", 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728273+0000", 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728274+0000", 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6899999999999996e-07 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728279+0000", 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0270000000000003e-06 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728293+0000", 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3900000000000001e-05 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728300+0000", 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1640000000000004e-06 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=26, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.728556+0000", 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.048603252, 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.049315061, 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.804 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728556+0000", 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728556+0000", 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728557+0000", 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5000000000000001e-07 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728563+0000", 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2299999999999999e-06 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728578+0000", 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5582000000000001e-05 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728587+0000", 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7539999999999995e-06 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=26, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.728727+0000", 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.048432121000000002, 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.049214641000000003, 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728727+0000", 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728727+0000", 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728728+0000", 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3600000000000003e-07 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728732+0000", 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2089999999999999e-06 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728746+0000", 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3583e-05 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728753+0000", 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4250000000000001e-06 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.810 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=26, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.728838+0000", 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.048321229, 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.049172088000000003, 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "no flag points reached", 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728838+0000", 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728838+0000", 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728839+0000", 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6700000000000003e-07 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728845+0000", 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.553e-06 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.728858+0000", 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3149000000000001e-05 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=26, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.729248+0000", 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.047911469999999998, 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.048823110000000003, 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:21.813 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.729248+0000", 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.729248+0000", 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.729249+0000", 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1699999999999998e-07 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.729252+0000", 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3749999999999999e-06 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.729262+0000", 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9820000000000003e-06 2024-04-18T23:11:21.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.729276+0000", 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4462e-05 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 11 2024-04-18T23:11:21.816 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:21.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:11:21.819 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:21.816+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:21.919 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:21.918+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:21.936 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:21.950 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:11:22.020 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:22.018+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:22.074 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:22.074 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 33/31 ECSubWrite(tid=1, reqid=client.4568.0:16, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 25.555500037000002, 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104807236, 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.075 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6599999999999996e-07 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.076 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517909+0000", 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013603000000000001 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517960+0000", 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1122000000000003e-05 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.618057+0000", 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10009630999999999 2024-04-18T23:11:22.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.618206+0000", 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000149259 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.618219+0000", 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2761e-05 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.078 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.622544+0000", 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043254590000000002 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.622580+0000", 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5729000000000003e-05 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.079 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "pg_update_log_missing(3.9s2 epoch 37/31 rep_tid 41 entries 37'17 (0'0) error 3:9e84ac80:::smithi14230322-40:head by client.4568.0:214 0.000000 0 ObjectCleanRegions clean_offsets: [(0, 18446744073709551615)], clean_omap: true, new_object: false trim_to 0'0 roll_forward_to 35'15)", 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.646387+0000", 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 20.426886052, 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10419487700000001, 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646387+0000", 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.080 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646387+0000", 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646387+0000", 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7700000000000005e-07 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646405+0000", 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7722000000000001e-05 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.081 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.646461+0000", 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6085999999999998e-05 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.746622+0000", 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10016080400000001 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.082 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.749532+0000", 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029097210000000001 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.750582+0000", 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001050067 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.5s5 40/31 ECSubWrite(tid=27, reqid=osd.15.0:26, at_version=0'0, trim_to=0'0, roll_forward_to=40'15))", 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:04.973085+0000", 2024-04-18T23:11:22.083 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 17.100187338000001, 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.11391169600000001, 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973085+0000", 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973085+0000", 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973086+0000", 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2099999999999995e-07 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973103+0000", 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7042999999999999e-05 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.085 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973153+0000", 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.003e-05 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973166+0000", 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2340999999999999e-05 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086086+0000", 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.112919777 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.086 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086100+0000", 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4949999999999999e-05 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086924+0000", 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082378300000000002 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086997+0000", 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2850999999999998e-05 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 40/31 ECSubWrite(tid=81, reqid=osd.4.0:79, at_version=0'0, trim_to=0'0, roll_forward_to=40'33))", 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:04.973664+0000", 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 17.099608453999998, 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102955542, 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973664+0000", 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973664+0000", 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973666+0000", 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0099999999999998e-06 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973686+0000", 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9536000000000001e-05 2024-04-18T23:11:22.089 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973723+0000", 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7302000000000003e-05 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973744+0000", 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0993e-05 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.090 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.073978+0000", 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10023391399999999 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.073996+0000", 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7612999999999999e-05 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.076565+0000", 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025693119999999998 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.091 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.076620+0000", 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4861999999999998e-05 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 40/31 ECSubWrite(tid=82, reqid=osd.4.0:82, at_version=40'35, trim_to=0'0, roll_forward_to=40'33))", 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:04.974058+0000", 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 17.099214897, 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10443377099999999, 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.092 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974058+0000", 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974058+0000", 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.093 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974059+0000", 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7599999999999996e-07 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974104+0000", 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5636000000000001e-05 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.974120+0000", 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5654999999999998e-05 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.094 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.074238+0000", 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10011843199999999 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.074461+0000", 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022277000000000001 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.074485+0000", 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3546000000000001e-05 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.095 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.078389+0000", 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0039044829999999998 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.078492+0000", 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000102373 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.096 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.8s1 40/31 ECSubWrite(tid=19, reqid=osd.5.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:05.187735+0000", 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 16.885538043, 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201244059, 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187735+0000", 2024-04-18T23:11:22.097 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187735+0000", 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187741+0000", 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4200000000000004e-06 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.098 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187762+0000", 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1095999999999999e-05 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187791+0000", 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8875999999999999e-05 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.287866+0000", 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100074872 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388009+0000", 2024-04-18T23:11:22.099 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100142971 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388016+0000", 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5919999999999997e-06 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388929+0000", 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00091366799999999997 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388979+0000", 2024-04-18T23:11:22.100 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9564000000000002e-05 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 42/31 ECSubWrite(tid=89, reqid=client.4568.0:279, at_version=42'41, trim_to=0'0, roll_forward_to=42'39))", 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:07.091573+0000", 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.981700085, 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20162514400000001, 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.101 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091573+0000", 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091573+0000", 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.102 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091575+0000", 2024-04-18T23:11:22.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0229999999999999e-06 2024-04-18T23:11:22.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.103 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.103 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.103 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091804+0000", 2024-04-18T23:11:22.103 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022908800000000001 2024-04-18T23:11:22.103 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.104 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.104 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091827+0000", 2024-04-18T23:11:22.105 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3210999999999999e-05 2024-04-18T23:11:22.105 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.105 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.105 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.105 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091839+0000", 2024-04-18T23:11:22.106 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2091e-05 2024-04-18T23:11:22.106 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.106 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.106 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.292067+0000", 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20022828100000001 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.292074+0000", 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8379999999999996e-06 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.293170+0000", 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010957250000000001 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.293198+0000", 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7886999999999999e-05 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs6 51/31 ECSubWrite(tid=23, reqid=client.4568.0:384, at_version=51'4, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.771552+0000", 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.3017207989999999, 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103458705, 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.108 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.771552+0000", 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.771552+0000", 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.771553+0000", 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:22.109 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.772000+0000", 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00044736899999999998 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.772033+0000", 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3129000000000001e-05 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.110 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.772063+0000", 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9876e-05 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.872340+0000", 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100277414 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.872370+0000", 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9407000000000002e-05 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.111 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.874922+0000", 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025524020000000001 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.875011+0000", 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8457999999999999e-05 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs6 51/31 ECSubWrite(tid=24, reqid=client.4568.0:385, at_version=51'5, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.776073+0000", 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.2972002060000003, 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.203822843, 2024-04-18T23:11:22.112 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776073+0000", 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776073+0000", 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.113 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776074+0000", 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6899999999999999e-06 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776440+0000", 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036623500000000002 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776490+0000", 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9808999999999997e-05 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.114 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876611+0000", 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100120291 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.977075+0000", 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100464015 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.977108+0000", 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3593000000000002e-05 2024-04-18T23:11:22.115 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.979820+0000", 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027116419999999998 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.979895+0000", 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5568000000000006e-05 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs6 51/31 ECSubWrite(tid=25, reqid=client.4568.0:386, at_version=51'6, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:22.116 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.876665+0000", 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.1966075519999997, 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.1052046, 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876665+0000", 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876665+0000", 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.117 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876667+0000", 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4360000000000001e-06 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876763+0000", 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6008000000000006e-05 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876795+0000", 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2045999999999998e-05 2024-04-18T23:11:22.118 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876831+0000", 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6579000000000003e-05 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.978579+0000", 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101747526 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.978598+0000", 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9477e-05 2024-04-18T23:11:22.119 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981785+0000", 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00318651 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981870+0000", 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5018000000000005e-05 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.120 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs6 51/31 ECSubWrite(tid=26, reqid=client.4568.0:387, at_version=51'7, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.876865+0000", 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.1964075369999998, 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.105074846, 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876865+0000", 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.121 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876865+0000", 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876866+0000", 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2600000000000001e-07 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876899+0000", 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3246e-05 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.122 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876923+0000", 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3563000000000001e-05 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876941+0000", 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8023e-05 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.979369+0000", 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10242828599999999 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.979399+0000", 2024-04-18T23:11:22.123 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9759000000000001e-05 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981877+0000", 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002478175 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981940+0000", 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2867999999999995e-05 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.124 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds4 51/31 ECSubWrite(tid=50, reqid=client.4568.0:389, at_version=51'31, trim_to=0'0, roll_forward_to=47'29))", 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.877090+0000", 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.1961832169999997, 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104646974, 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877090+0000", 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.125 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877090+0000", 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877090+0000", 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000001e-07 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877352+0000", 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00026140700000000001 2024-04-18T23:11:22.126 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877379+0000", 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7902000000000002e-05 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877401+0000", 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1440000000000001e-05 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.977596+0000", 2024-04-18T23:11:22.127 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10019465900000001 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.977611+0000", 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5583e-05 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981660+0000", 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040493580000000003 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981737+0000", 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6070999999999997e-05 2024-04-18T23:11:22.128 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds4 51/31 ECSubWrite(tid=51, reqid=client.4568.0:390, at_version=51'32, trim_to=0'0, roll_forward_to=47'29))", 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.878002+0000", 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.1952710059999996, 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103848814, 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.878002+0000", 2024-04-18T23:11:22.129 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.878002+0000", 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.878004+0000", 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8360000000000001e-06 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.878291+0000", 2024-04-18T23:11:22.130 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028750299999999999 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.878330+0000", 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8563000000000003e-05 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.878360+0000", 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0611999999999998e-05 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.978917+0000", 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100557067 2024-04-18T23:11:22.131 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.978943+0000", 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5885999999999999e-05 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981814+0000", 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028708179999999998 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981851+0000", 2024-04-18T23:11:22.132 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6529000000000002e-05 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:416 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~524288] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.517140+0000", 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.55613243599999995, 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.21039138299999999, 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.133 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 416 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517140+0000", 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517140+0000", 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.134 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517143+0000", 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5900000000000002e-06 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517802+0000", 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000659215 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517830+0000", 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7472999999999999e-05 2024-04-18T23:11:22.135 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517843+0000", 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3261000000000001e-05 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618107+0000", 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100263651 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618120+0000", 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3016999999999999e-05 2024-04-18T23:11:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.619287+0000", 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011671710000000001 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.619457+0000", 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016981799999999999 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.621037+0000", 2024-04-18T23:11:22.137 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001580084 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727045+0000", 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10600874 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727341+0000", 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029538900000000001 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727532+0000", 2024-04-18T23:11:22.138 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000190974 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:417 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~524288 in=524288b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.55316986800000001, 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30835631899999999, 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.139 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 417 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.140 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520104+0000", 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2389999999999999e-06 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520670+0000", 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00056593599999999996 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520690+0000", 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9945e-05 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.141 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520699+0000", 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0429999999999996e-06 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618217+0000", 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097518176999999998 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618227+0000", 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7860000000000008e-06 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.142 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622364+0000", 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041371389999999997 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622653+0000", 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028858600000000002 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.624345+0000", 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001691782 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.143 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727165+0000", 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102820087 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828093+0000", 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100928079 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828459+0000", 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036652000000000001 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.144 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:418 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~245760 in=245760b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.55168701200000003, 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30828365299999999, 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 418 2024-04-18T23:11:22.145 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521587+0000", 2024-04-18T23:11:22.146 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2100000000000001e-06 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521960+0000", 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000373402 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521972+0000", 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.207e-05 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.147 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521979+0000", 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7839999999999998e-06 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725251+0000", 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20327163200000001 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725428+0000", 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000177071 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.726423+0000", 2024-04-18T23:11:22.148 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00099555200000000003 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828646+0000", 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102223007 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829746+0000", 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010997509999999999 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.149 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829869+0000", 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012317400000000001 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 55/49 ECSubWriteReply(tid=24, last_complete=56'17, committed=1, applied=1))", 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.623375+0000", 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.44989743500000001, 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103739574, 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.150 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623375+0000", 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623375+0000", 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623387+0000", 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.145e-05 2024-04-18T23:11:22.151 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623395+0000", 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6770000000000001e-06 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623416+0000", 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0908999999999999e-05 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623434+0000", 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7952999999999999e-05 2024-04-18T23:11:22.152 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727099+0000", 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103664521 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727115+0000", 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6064000000000001e-05 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=24, last_complete=56'17, committed=1, applied=1))", 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.623874+0000", 2024-04-18T23:11:22.153 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.44939899700000002, 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103669705, 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623874+0000", 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623874+0000", 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.154 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623875+0000", 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.173e-06 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623881+0000", 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0970000000000002e-06 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623899+0000", 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8063e-05 2024-04-18T23:11:22.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623910+0000", 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.078e-05 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727278+0000", 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10336791200000001 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727543+0000", 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00026568000000000002 2024-04-18T23:11:22.156 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 55/49 ECSubWriteReply(tid=24, last_complete=56'17, committed=1, applied=1))", 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.623973+0000", 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.44930002000000002, 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103193969, 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623973+0000", 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623973+0000", 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623974+0000", 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5570000000000001e-06 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.623985+0000", 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1017e-05 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.624010+0000", 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4485e-05 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.624031+0000", 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0757000000000001e-05 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727150+0000", 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103119192 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727167+0000", 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6960999999999999e-05 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=25, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.627081+0000", 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.44619138400000002, 2024-04-18T23:11:22.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20139538000000001, 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627081+0000", 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627081+0000", 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627083+0000", 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.443e-06 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627091+0000", 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0469999999999994e-06 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627114+0000", 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2679000000000001e-05 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627125+0000", 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1513e-05 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.827940+0000", 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20081492200000001 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828477+0000", 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053677599999999998 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.164 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.164 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.164 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:22.165 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:11:22.166 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:22.162+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:22.266 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:22.266+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:22.335 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:22.335 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:22.335 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:11:22.335 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:22.349 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:11:22.367 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:22.364+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:22.468 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:22.466+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:22.472 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:22.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:11:22.519 INFO:tasks.rados.rados.0.smithi142.stdout:173: done (5 left) 2024-04-18T23:11:22.519 INFO:tasks.rados.rados.0.smithi142.stdout:174: delete oid 18 current snap is 14 2024-04-18T23:11:22.519 INFO:tasks.rados.rados.0.smithi142.stdout:172: finishing copy_from racing read to smithi14230322-50 2024-04-18T23:11:22.519 INFO:tasks.rados.rados.0.smithi142.stdout:168: finishing write tid 1 to smithi14230322-31 2024-04-18T23:11:22.519 INFO:tasks.rados.rados.0.smithi142.stdout:168: finishing write tid 2 to smithi14230322-31 2024-04-18T23:11:22.519 INFO:tasks.rados.rados.0.smithi142.stdout:168: finishing write tid 3 to smithi14230322-31 2024-04-18T23:11:22.519 INFO:tasks.rados.rados.0.smithi142.stdout:172: finishing copy_from to smithi14230322-50 2024-04-18T23:11:22.519 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 50 v 2 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-04-18T23:11:22.520 INFO:tasks.rados.rados.0.smithi142.stdout:171: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:22.521 INFO:tasks.rados.rados.0.smithi142.stdout:170: finishing copy_from racing read to smithi14230322-10 2024-04-18T23:11:22.521 INFO:tasks.rados.rados.0.smithi142.stdout:170: finishing copy_from to smithi14230322-10 2024-04-18T23:11:22.521 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 10 v 16 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-18T23:11:22.521 INFO:tasks.rados.rados.0.smithi142.stdout:168: finishing write tid 4 to smithi14230322-31 2024-04-18T23:11:22.521 INFO:tasks.rados.rados.0.smithi142.stdout:169: finishing write tid 1 to smithi14230322-16 2024-04-18T23:11:22.521 INFO:tasks.rados.rados.0.smithi142.stdout:169: finishing write tid 2 to smithi14230322-16 2024-04-18T23:11:22.521 INFO:tasks.rados.rados.0.smithi142.stdout:169: finishing write tid 3 to smithi14230322-16 2024-04-18T23:11:22.549 INFO:tasks.rados.rados.0.smithi142.stdout:169: finishing write tid 4 to smithi14230322-16 2024-04-18T23:11:22.569 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:22.568+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:22.617 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:21 3.bs0 3:d498df52:::smithi14230322-7:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.517296+0000", 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.097029536000001, 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.31922700999999998, 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.618 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 21 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517296+0000", 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517296+0000", 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.619 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517297+0000", 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3859999999999999e-06 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517303+0000", 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.237e-06 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517327+0000", 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4263000000000001e-05 2024-04-18T23:11:22.620 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517342+0000", 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4806e-05 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628177+0000", 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.110835323 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628320+0000", 2024-04-18T23:11:22.621 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014254200000000001 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.825855+0000", 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.197534974 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.826053+0000", 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000198927 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.836523+0000", 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.010469552 2024-04-18T23:11:22.622 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:181 3.bs0 3:dd01ba5f:::smithi14230322-23:head [append 0~524288] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.552123+0000", 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.062202143, 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.31423346800000002, 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.623 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 181 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552123+0000", 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552123+0000", 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.624 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552126+0000", 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6690000000000002e-06 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552796+0000", 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00066951100000000002 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552835+0000", 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9122000000000003e-05 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.625 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552847+0000", 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2509e-05 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552926+0000", 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8658000000000005e-05 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.552932+0000", 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8780000000000003e-06 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.626 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.657708+0000", 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10477632200000001 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.657762+0000", 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3603000000000002e-05 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658408+0000", 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064599500000000001 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.627 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757880+0000", 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099471941999999994 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.866304+0000", 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.108424836 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.866357+0000", 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2423000000000003e-05 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.628 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:183 3.bs0 3:dd01ba5f:::smithi14230322-23:head [setxattr _header (56) in=7b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.557454+0000", 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.056871197, 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40969022799999999, 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 183 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.629 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557454+0000", 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557454+0000", 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557455+0000", 2024-04-18T23:11:22.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.89e-07 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557459+0000", 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8070000000000001e-06 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557466+0000", 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9e-06 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557470+0000", 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4440000000000001e-06 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557529+0000", 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8903999999999997e-05 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557532+0000", 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6579999999999998e-06 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.659801+0000", 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102268964 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.659849+0000", 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8466e-05 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.660148+0000", 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029888700000000001 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757882+0000", 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097734330999999994 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967050+0000", 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20916807400000001 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967145+0000", 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4303999999999997e-05 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.634 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:184 3.bs0 3:dd01ba5f:::smithi14230322-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.056834635000001, 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61588253999999998, 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 184 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.635 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:22.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8700000000000001e-07 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557493+0000", 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9149999999999999e-06 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557499+0000", 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2650000000000002e-06 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557504+0000", 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5689999999999998e-06 2024-04-18T23:11:22.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654492+0000", 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.096987633000000004 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654498+0000", 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5860000000000001e-06 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757767+0000", 2024-04-18T23:11:22.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103269131 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757839+0000", 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1878999999999994e-05 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967626+0000", 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.209786891 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967877+0000", 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025035699999999999 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173373+0000", 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.205496927 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:188 3.7s0 3:e5b49aa4:::smithi14230322-32:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558569+0000", 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.05575627, 2024-04-18T23:11:22.640 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50916866999999999, 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 188 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558569+0000", 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.641 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558569+0000", 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558570+0000", 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2499999999999995e-07 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558621+0000", 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1082999999999999e-05 2024-04-18T23:11:22.642 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558629+0000", 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9529999999999999e-06 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558634+0000", 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7360000000000001e-06 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558665+0000", 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1099000000000002e-05 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.643 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558669+0000", 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3610000000000003e-06 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654665+0000", 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.095996355000000005 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654801+0000", 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013551599999999999 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.644 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.655971+0000", 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011700059999999999 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.657081+0000", 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001109737 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067667+0000", 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41058618899999999 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.645 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067738+0000", 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-05 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:189 3.bs0 3:d498df52:::smithi14230322-7:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.055642353, 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.91155819000000005, 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.646 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 189 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.647 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558684+0000", 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1799999999999995e-07 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558687+0000", 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1920000000000001e-06 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.648 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558694+0000", 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.452e-06 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658767+0000", 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100072129 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757920+0000", 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099153626999999994 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.649 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757967+0000", 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6962999999999997e-05 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171274+0000", 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41330696700000003 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173905+0000", 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002630828 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.650 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470100+0000", 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29619521399999998 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470241+0000", 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014119999999999999 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:190 3.bs0 3:d498df52:::smithi14230322-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.658782+0000", 2024-04-18T23:11:22.651 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 21.955543503000001, 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.51341810700000001, 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 190 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658782+0000", 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658782+0000", 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658783+0000", 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1209999999999999e-06 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658789+0000", 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-04-18T23:11:22.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658798+0000", 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6099999999999995e-06 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658805+0000", 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4489999999999996e-06 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760577+0000", 2024-04-18T23:11:22.654 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101772846 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760624+0000", 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6789000000000002e-05 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.172000+0000", 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41137571899999997 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.655 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.172095+0000", 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4592000000000005e-05 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.172200+0000", 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105546 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:196 3.7s0 3:e60e3c0c:::smithi14230322-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.658830+0000", 2024-04-18T23:11:22.656 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 21.955495662000001, 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40905263600000002, 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 196 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658830+0000", 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658830+0000", 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658831+0000", 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.37e-07 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658834+0000", 2024-04-18T23:11:22.658 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4589999999999998e-06 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658841+0000", 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2629999999999997e-06 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758909+0000", 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100068197 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.659 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758967+0000", 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7574999999999997e-05 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759068+0000", 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101267 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759746+0000", 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00067788199999999999 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.660 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863226+0000", 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103479856 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067834+0000", 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20460830399999999 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067883+0000", 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8195999999999997e-05 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:197 3.7s0 3:e60e3c0c:::smithi14230322-6:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.758927+0000", 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 21.855399038000002, 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40981388400000002, 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 197 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.662 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758927+0000", 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758927+0000", 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758928+0000", 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0470000000000001e-06 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758932+0000", 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758939+0000", 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9280000000000004e-06 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758944+0000", 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4700000000000004e-06 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760155+0000", 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001211073 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760309+0000", 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015378399999999999 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760963+0000", 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00065410399999999997 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.665 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863295+0000", 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102332388 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168687+0000", 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30539211500000002 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168740+0000", 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3152999999999997e-05 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.666 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:198 3.7s0 3:e60e3c0c:::smithi14230322-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 21.855367911999998, 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41079902800000001, 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 198 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9799999999999999e-07 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.668 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758960+0000", 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.623e-06 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758964+0000", 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4819999999999996e-06 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758967+0000", 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4570000000000001e-06 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761240+0000", 2024-04-18T23:11:22.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002272081 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761347+0000", 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010780099999999999 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761749+0000", 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00040184299999999998 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863298+0000", 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101549268 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169722+0000", 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.306423995 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169757+0000", 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4180000000000001e-05 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:212 3.7s0 3:e405d834:::smithi14230322-14:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 21.855352573000001, 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41097265900000002, 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 212 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9099999999999999e-07 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759003+0000", 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9641999999999998e-05 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759008+0000", 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.271e-06 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759012+0000", 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1359999999999999e-06 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.862010+0000", 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102997908 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.862080+0000", 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9305000000000001e-05 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.862724+0000", 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064440199999999997 2024-04-18T23:11:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863536+0000", 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081221099999999997 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169861+0000", 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30632486799999997 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169946+0000", 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4524999999999995e-05 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:279 3.7s0 3:ea1bab24:::smithi14230322-36:head [copy-from ver 15] snapc 6=[6,5,4] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.579501+0000", 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 16.034824436000001, 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.81417189400000001, 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 279 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579501+0000", 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579501+0000", 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579503+0000", 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.798e-06 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579508+0000", 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4219999999999999e-06 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579520+0000", 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1833e-05 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579531+0000", 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1080999999999999e-05 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677050+0000", 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097518821000000006 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677182+0000", 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013219100000000001 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091592+0000", 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41440982100000001 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.095981+0000", 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043885729999999998 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.393572+0000", 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29759129499999998 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.393673+0000", 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101059 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:280 3.7s0 3:ea1bab24:::smithi14230322-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.579552+0000", 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 16.034773842, 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.51371551100000001, 2024-04-18T23:11:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 280 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579552+0000", 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579552+0000", 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579552+0000", 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8500000000000001e-07 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579555+0000", 2024-04-18T23:11:22.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6390000000000001e-06 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579564+0000", 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1130000000000007e-06 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579571+0000", 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5570000000000001e-06 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677388+0000", 2024-04-18T23:11:22.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097817297999999997 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677445+0000", 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7309999999999998e-05 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.093070+0000", 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41562443500000001 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.093174+0000", 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104144 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.093267+0000", 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3430000000000005e-05 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:322 3.bs0 3:d447fef0:::smithi14230322-49:8 [read 0~819200,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:10.608558+0000", 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.005767551, 2024-04-18T23:11:22.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.44893157099999997, 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 322 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608558+0000", 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608558+0000", 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608559+0000", 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3300000000000002e-07 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608562+0000", 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8029999999999998e-06 2024-04-18T23:11:22.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608572+0000", 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8749999999999995e-06 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608580+0000", 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1089999999999998e-06 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608674+0000", 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4075999999999998e-05 2024-04-18T23:11:22.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608678+0000", 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5860000000000001e-06 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.913531+0000", 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30485333599999997 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.913701+0000", 2024-04-18T23:11:22.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016934599999999999 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.057490+0000", 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.14378880699999999 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:324 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~0] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:11.423978+0000", 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.190348008999999, 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41043667700000003, 2024-04-18T23:11:22.692 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 324 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.423978+0000", 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.423978+0000", 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.423980+0000", 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9000000000000002e-06 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.423992+0000", 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1823e-05 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424039+0000", 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6909000000000001e-05 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424056+0000", 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6762999999999999e-05 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424196+0000", 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000140063 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424379+0000", 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018270600000000001 2024-04-18T23:11:22.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.426794+0000", 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002415122 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.432886+0000", 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060920269999999999 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.834286+0000", 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40140048 2024-04-18T23:11:22.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.834414+0000", 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012788400000000001 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:325 3.bs0 3:d447fef0:::smithi14230322-49:head [rollback 7] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:11.424087+0000", 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.190238838000001, 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41134601399999998, 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 325 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424087+0000", 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424087+0000", 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424088+0000", 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3230000000000001e-06 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424100+0000", 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1559999999999999e-05 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424119+0000", 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9341999999999999e-05 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424134+0000", 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5284000000000001e-05 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.428226+0000", 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040921229999999996 2024-04-18T23:11:22.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.428477+0000", 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025047399999999999 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.429867+0000", 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013900550000000001 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.432891+0000", 2024-04-18T23:11:22.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030243679999999999 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.835231+0000", 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40233970299999999 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.835433+0000", 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020178199999999999 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:326 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~0] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:11.424155+0000", 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.190170333999999, 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.51120384900000004, 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 326 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.703 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424155+0000", 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424155+0000", 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424156+0000", 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.237e-06 2024-04-18T23:11:22.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424162+0000", 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.057e-06 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424179+0000", 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7405999999999999e-05 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424192+0000", 2024-04-18T23:11:22.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3553000000000001e-05 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.430657+0000", 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0064646080000000002 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.430911+0000", 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025408400000000002 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.432114+0000", 2024-04-18T23:11:22.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012027559999999999 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.433102+0000", 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00098769500000000002 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935230+0000", 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50212803900000003 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935359+0000", 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000129414 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:334 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~704512] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.636133+0000", 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9781925010000005, 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50100039200000002, 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.708 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 334 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.636133+0000", 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.636133+0000", 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.636136+0000", 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6410000000000002e-06 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.637067+0000", 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093128099999999995 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.637100+0000", 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3339e-05 2024-04-18T23:11:22.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.637118+0000", 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7377999999999999e-05 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.829485+0000", 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19236683600000001 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.829658+0000", 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017355300000000001 2024-04-18T23:11:22.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.831259+0000", 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0016010309999999999 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.934664+0000", 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103405119 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.137039+0000", 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20237438399999999 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.137133+0000", 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4829999999999998e-05 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:335 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~376832] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.638777+0000", 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9755488400000001, 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.49869328600000001, 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.713 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 335 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.638777+0000", 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.638777+0000", 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.638779+0000", 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3309999999999998e-06 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639167+0000", 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038792399999999999 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639190+0000", 2024-04-18T23:11:22.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3383e-05 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639204+0000", 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3715e-05 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.932912+0000", 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29370825099999998 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933100+0000", 2024-04-18T23:11:22.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018750599999999999 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933966+0000", 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00086649399999999999 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.136557+0000", 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20259101099999999 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.137391+0000", 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083378900000000004 2024-04-18T23:11:22.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.137470+0000", 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8881999999999995e-05 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:336 3.bs0 3:d447fef0:::smithi14230322-49:head [setxattr _header (56) in=7b] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.639228+0000", 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9750977429999992, 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.59823371800000003, 2024-04-18T23:11:22.718 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 336 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639228+0000", 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639228+0000", 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639229+0000", 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6599999999999994e-07 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639234+0000", 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639246+0000", 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2208000000000001e-05 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639254+0000", 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5469999999999999e-06 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.033282+0000", 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39402777999999999 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.033456+0000", 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017395900000000001 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.034378+0000", 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00092277099999999996 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.136559+0000", 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102180697 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.237346+0000", 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100786389 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.237462+0000", 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115977 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.723 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:22.724 INFO:tasks.rados.rados.0.smithi142.stdout:168: finishing write tid 5 to smithi14230322-31 2024-04-18T23:11:22.724 INFO:tasks.rados.rados.0.smithi142.stdout:168: oid 31 updating version 0 to 20 2024-04-18T23:11:22.725 INFO:tasks.rados.rados.0.smithi142.stdout:168: oid 31 version 20 is already newer than 19 2024-04-18T23:11:22.725 INFO:tasks.rados.rados.0.smithi142.stdout:168: oid 31 version 20 is already newer than 17 2024-04-18T23:11:22.725 INFO:tasks.rados.rados.0.smithi142.stdout:168: oid 31 version 20 is already newer than 18 2024-04-18T23:11:22.725 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 31 v 20 (ObjNum 77 snap 14 seq_num 77) dirty exists 2024-04-18T23:11:22.725 INFO:tasks.rados.rados.0.smithi142.stdout:168: left oid 31 (ObjNum 77 snap 14 seq_num 77) 2024-04-18T23:11:22.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:11:22.726 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:22.724+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:22.727 INFO:tasks.rados.rados.0.smithi142.stdout:168: done (5 left) 2024-04-18T23:11:22.727 INFO:tasks.rados.rados.0.smithi142.stdout:170: done (4 left) 2024-04-18T23:11:22.727 INFO:tasks.rados.rados.0.smithi142.stdout:171: done (3 left) 2024-04-18T23:11:22.727 INFO:tasks.rados.rados.0.smithi142.stdout:172: done (2 left) 2024-04-18T23:11:22.727 INFO:tasks.rados.rados.0.smithi142.stdout:174: done (1 left) 2024-04-18T23:11:22.727 INFO:tasks.rados.rados.0.smithi142.stdout:175: snap_remove snap 9 2024-04-18T23:11:22.826 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:22.822+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:22.859 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:22.859 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:22.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.859 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/31 ECSubWriteReply(tid=53, last_complete=57'31, committed=1, applied=1))", 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.829387+0000", 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.030240678, 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.030276107, 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.829387+0000", 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.829387+0000", 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.829390+0000", 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6460000000000002e-06 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.829403+0000", 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.327e-05 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.829436+0000", 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2869e-05 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.829454+0000", 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7779999999999999e-05 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.862 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 57/31 ECSubWriteReply(tid=53, last_complete=57'31, committed=1, applied=1))", 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.831209+0000", 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.02841817, 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.028536156999999999, 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.831209+0000", 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.831209+0000", 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.831211+0000", 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8700000000000001e-06 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.831221+0000", 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0013000000000001e-05 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.831249+0000", 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7416e-05 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.831262+0000", 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3699e-05 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s4 57/31 ECSubWrite(tid=170, reqid=osd.4.0:167, at_version=0'0, trim_to=0'0, roll_forward_to=57'77))", 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.832355+0000", 2024-04-18T23:11:22.865 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.027272606000000001, 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.027466981000000001, 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.832355+0000", 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.832355+0000", 2024-04-18T23:11:22.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.832357+0000", 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8530000000000001e-06 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.832383+0000", 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6599000000000001e-05 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.832417+0000", 2024-04-18T23:11:22.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3785999999999998e-05 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.832440+0000", 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2802999999999998e-05 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs4 57/31 ECSubWrite(tid=34, reqid=osd.11.0:33, at_version=0'0, trim_to=0'0, roll_forward_to=57'11))", 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.837617+0000", 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.022010210999999998, 2024-04-18T23:11:22.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.022277643999999999, 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.837617+0000", 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.837617+0000", 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:22.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.837619+0000", 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.246e-06 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.837640+0000", 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1325999999999999e-05 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.837674+0000", 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4239999999999997e-05 2024-04-18T23:11:22.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.837694+0000", 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9709000000000001e-05 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 4 2024-04-18T23:11:22.871 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:22.873 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:11:22.960 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:22.958+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:23.028 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:23.024+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:23.038 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:23.052 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:11:23.129 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:23.130+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:23.226 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:23.226 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:23.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:23.226 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:23.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.226 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds5 34/31 ECSubWrite(tid=0, reqid=client.4568.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.532278+0000", 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.68893014, 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20155427200000001, 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532278+0000", 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532278+0000", 2024-04-18T23:11:23.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532278+0000", 2024-04-18T23:11:23.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7500000000000001e-07 2024-04-18T23:11:23.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532500+0000", 2024-04-18T23:11:23.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022245699999999999 2024-04-18T23:11:23.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532515+0000", 2024-04-18T23:11:23.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4592e-05 2024-04-18T23:11:23.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.532527+0000", 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2218999999999999e-05 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.632678+0000", 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100150425 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.632686+0000", 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7609999999999996e-06 2024-04-18T23:11:23.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.733761+0000", 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10107475 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.733832+0000", 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0692999999999996e-05 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:23.231 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.628124+0000", 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.593083371999999, 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20045701599999999, 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628124+0000", 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628124+0000", 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628125+0000", 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2200000000000002e-07 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628131+0000", 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2530000000000004e-06 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.628145+0000", 2024-04-18T23:11:23.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4430000000000001e-05 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728230+0000", 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100084688 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.828536+0000", 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10030650100000001 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.828581+0000", 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5222000000000002e-05 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=2, last_complete=34'3, committed=1, applied=1))", 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.728315+0000", 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.49289285, 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20072921299999999, 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728315+0000", 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728315+0000", 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728316+0000", 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1600000000000001e-07 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728321+0000", 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9799999999999998e-06 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728339+0000", 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8584e-05 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728352+0000", 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3124e-05 2024-04-18T23:11:23.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.929018+0000", 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20066600000000001 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.929044+0000", 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5908999999999999e-05 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.238 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.729293+0000", 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.49191523, 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19966805300000001, 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729293+0000", 2024-04-18T23:11:23.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729293+0000", 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729293+0000", 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8400000000000003e-07 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729302+0000", 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5620000000000004e-06 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729315+0000", 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3263e-05 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729327+0000", 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1592e-05 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.928932+0000", 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.199605167 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.928961+0000", 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8585000000000002e-05 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.242 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 33/31 ECSubWriteReply(tid=2, last_complete=34'3, committed=1, applied=1))", 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.729347+0000", 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.491860990999999, 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29994712699999998, 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729347+0000", 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729347+0000", 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729347+0000", 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5000000000000003e-07 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729351+0000", 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8480000000000003e-06 2024-04-18T23:11:23.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729364+0000", 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3171e-05 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.729374+0000", 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1779999999999992e-06 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.029281+0000", 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29990735099999999 2024-04-18T23:11:23.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.029294+0000", 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3029e-05 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=0, last_complete=34'1, committed=1, applied=1))", 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.732253+0000", 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.488955143999998, 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196610745, 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.246 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732253+0000", 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732253+0000", 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732254+0000", 2024-04-18T23:11:23.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2920000000000001e-06 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732262+0000", 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6320000000000008e-06 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732285+0000", 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3434e-05 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732300+0000", 2024-04-18T23:11:23.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5390999999999999e-05 2024-04-18T23:11:23.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.928829+0000", 2024-04-18T23:11:23.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19652907 2024-04-18T23:11:23.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.928863+0000", 2024-04-18T23:11:23.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3926e-05 2024-04-18T23:11:23.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.250 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 34/31 ECSubWriteReply(tid=1, last_complete=34'2, committed=1, applied=1))", 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.732325+0000", 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.488882880999999, 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29692952, 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732325+0000", 2024-04-18T23:11:23.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732325+0000", 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732325+0000", 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7199999999999999e-07 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732330+0000", 2024-04-18T23:11:23.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3470000000000001e-06 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732343+0000", 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2816000000000001e-05 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.732352+0000", 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1600000000000004e-06 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.029234+0000", 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29688247800000001 2024-04-18T23:11:23.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.029254+0000", 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0146999999999999e-05 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:194 3.4s0 3:2a56ff57:::smithi14230322-24:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.662428436999999, 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41191497500000002, 2024-04-18T23:11:23.254 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 194 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558781+0000", 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8059999999999999e-06 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558789+0000", 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1340000000000004e-06 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558806+0000", 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6300999999999999e-05 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558814+0000", 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0099999999999995e-06 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558882+0000", 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8281999999999994e-05 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558889+0000", 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2359999999999998e-06 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661030+0000", 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102141155 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661144+0000", 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113853 2024-04-18T23:11:23.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766467+0000", 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.105323126 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.768352+0000", 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018843359999999999 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.970633+0000", 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.202281391 2024-04-18T23:11:23.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.970694+0000", 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1345000000000003e-05 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:195 3.4s0 3:2a56ff57:::smithi14230322-24:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558830+0000", 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.66237791, 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20893925299999999, 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.260 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 195 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558830+0000", 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558830+0000", 2024-04-18T23:11:23.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558830+0000", 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6699999999999999e-07 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558833+0000", 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9050000000000001e-06 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558838+0000", 2024-04-18T23:11:23.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5970000000000002e-06 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558848+0000", 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0351e-05 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558961+0000", 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011244299999999999 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558970+0000", 2024-04-18T23:11:23.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7229999999999999e-06 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661751+0000", 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102780887 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661782+0000", 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1164999999999997e-05 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.767573+0000", 2024-04-18T23:11:23.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10579037500000001 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.767662+0000", 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9672000000000005e-05 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.767769+0000", 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010676800000000001 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 36/31 ECSubWriteReply(tid=18, last_complete=36'11, committed=1, applied=1))", 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.972289+0000", 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.248919163, 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30040967099999999, 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972289+0000", 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972289+0000", 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972289+0000", 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8500000000000002e-07 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972292+0000", 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6970000000000002e-06 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.972298+0000", 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4110000000000001e-06 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.072395+0000", 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100096515 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.272633+0000", 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20023834400000001 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.272698+0000", 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5219e-05 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es4 42/31 ECSubWrite(tid=24, reqid=osd.2.0:24, at_version=42'14, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.593861+0000", 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 16.627346846999998, 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20227319099999999, 2024-04-18T23:11:23.269 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593861+0000", 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593861+0000", 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593861+0000", 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6899999999999999e-07 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593893+0000", 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2258999999999997e-05 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.593903+0000", 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2150000000000007e-06 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.693989+0000", 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008618499999999 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.795360+0000", 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10137168000000001 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.795374+0000", 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3842000000000001e-05 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.796103+0000", 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00072852400000000003 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.796134+0000", 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1316999999999998e-05 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.4.21:2 3.4s0 3:29b33eb5:::smithi14230322-2:head [assert-version v15,copy-get max 8388608 in=29b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.887375+0000", 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 16.333833168000002, 2024-04-18T23:11:23.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201454619, 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887375+0000", 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887375+0000", 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887376+0000", 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8279999999999999e-06 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887382+0000", 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9270000000000002e-06 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887395+0000", 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2757e-05 2024-04-18T23:11:23.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887404+0000", 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5420000000000005e-06 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887540+0000", 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013645399999999999 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.887655+0000", 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000114558 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.088829+0000", 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20117455300000001 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds5 47/31 ECSubWrite(tid=47, reqid=osd.7.0:47, at_version=47'27, trim_to=0'0, roll_forward_to=46'25))", 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.843706+0000", 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.377501935, 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201477294, 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.277 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.843706+0000", 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.843706+0000", 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.843706+0000", 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6100000000000002e-07 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.843731+0000", 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4363e-05 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.843748+0000", 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7552000000000001e-05 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.843760+0000", 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1593999999999999e-05 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.943952+0000", 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100192191 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.943962+0000", 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0027000000000001e-05 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.280 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.045086+0000", 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101124268 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.045183+0000", 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6937999999999999e-05 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.281 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds5 47/31 ECSubWrite(tid=48, reqid=osd.7.0:48, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.844561+0000", 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.376647078, 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20155680600000001, 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.844561+0000", 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.844561+0000", 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.844562+0000", 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.054e-06 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.844590+0000", 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7908000000000001e-05 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.844605+0000", 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5290000000000001e-05 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.844619+0000", 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4418e-05 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.044406+0000", 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.199786889 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.044422+0000", 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5563e-05 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.046048+0000", 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0016257960000000001 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.046118+0000", 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9888000000000006e-05 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.285 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs4 51/31 ECSubWrite(tid=23, reqid=client.4568.0:384, at_version=51'4, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.771818+0000", 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.4493897279999999, 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30501938899999997, 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.771818+0000", 2024-04-18T23:11:23.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.771818+0000", 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.771819+0000", 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7700000000000004e-07 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.772071+0000", 2024-04-18T23:11:23.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000252117 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.772122+0000", 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0988e-05 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.872277+0000", 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100154665 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.972546+0000", 2024-04-18T23:11:23.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100269264 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.972562+0000", 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6651999999999999e-05 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:17.076772+0000", 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104209521 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:17.076837+0000", 2024-04-18T23:11:23.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5505000000000004e-05 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs4 51/31 ECSubWrite(tid=24, reqid=client.4568.0:385, at_version=51'5, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.872338+0000", 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.3488700360000001, 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20455636899999999, 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.872338+0000", 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.872338+0000", 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.872339+0000", 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.243e-06 2024-04-18T23:11:23.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.872724+0000", 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038499699999999999 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.872761+0000", 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6662999999999998e-05 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.972901+0000", 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100140193 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.974104+0000", 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001203567 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.974122+0000", 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7275999999999999e-05 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:17.076844+0000", 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102722305 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:17.076894+0000", 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0124999999999999e-05 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:401 3.5s1 3:a6fe874e:::smithi14230322-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:18.504265+0000", 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.7169425340000002, 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30516069299999998, 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.294 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 401 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504265+0000", 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504265+0000", 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504266+0000", 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6799999999999999e-07 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504272+0000", 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1249999999999998e-06 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504282+0000", 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3640000000000005e-06 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504290+0000", 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6309999999999992e-06 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504377+0000", 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6607999999999995e-05 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504382+0000", 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6319999999999996e-06 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.608095+0000", 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103712123 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.608151+0000", 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6712000000000002e-05 2024-04-18T23:11:23.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.809079+0000", 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20092789899999999 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.809191+0000", 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112013 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.809426+0000", 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000234818 2024-04-18T23:11:23.299 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:434 3.4s0 3:2f7b52be:::smithi14230322-18:head [delete] snapc f=[f,d,c,b,9,7] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e57)", 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.520310+0000", 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.70089791400000001, 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20686338300000001, 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:23.300 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 434 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.520310+0000", 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.520310+0000", 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.520312+0000", 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3190000000000002e-06 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.520322+0000", 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4069999999999999e-06 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.520347+0000", 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5885e-05 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.520360+0000", 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2031000000000001e-05 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.520450+0000", 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0037999999999994e-05 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.520458+0000", 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6330000000000002e-06 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.523827+0000", 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033688189999999999 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.523971+0000", 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000144065 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.525072+0000", 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001101267 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.626286+0000", 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10121374700000001 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.727036+0000", 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10074973199999999 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.727173+0000", 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013744000000000001 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.305 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 56/31 ECSubWriteReply(tid=52, last_complete=57'31, committed=1, applied=1))", 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.528063+0000", 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.69314432400000003, 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.198714526, 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.528063+0000", 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.528063+0000", 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.528065+0000", 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.125e-06 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.307 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.528070+0000", 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6489999999999996e-06 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.528092+0000", 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1403000000000002e-05 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.528100+0000", 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3480000000000004e-06 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.308 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.726755+0000", 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19865470499999999 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.726778+0000", 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3295999999999999e-05 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 56/31 ECSubWriteReply(tid=52, last_complete=57'31, committed=1, applied=1))", 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.530410+0000", 2024-04-18T23:11:23.309 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.69079765400000004, 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.196300218, 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.530410+0000", 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.530410+0000", 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.310 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.530411+0000", 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2900000000000003e-07 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.530415+0000", 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5549999999999996e-06 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.530426+0000", 2024-04-18T23:11:23.311 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0461e-05 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.530432+0000", 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7660000000000004e-06 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.726679+0000", 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19624694500000001 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.726710+0000", 2024-04-18T23:11:23.312 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1761999999999998e-05 2024-04-18T23:11:23.313 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.313 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.313 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.313 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.313 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.313 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:23.315 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:11:23.315 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:23.310+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:23.415 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:23.410+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:23.472 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:23.484 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:23.484 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:23.484 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:11:23.484 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:23.499 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:11:23.516 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:23.514+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:23.527 INFO:tasks.rados.rados.0.smithi142.stdout:175: done (1 left) 2024-04-18T23:11:23.527 INFO:tasks.rados.rados.0.smithi142.stdout:176: snap_create 2024-04-18T23:11:23.527 INFO:tasks.rados.rados.0.smithi142.stdout:169: finishing write tid 5 to smithi14230322-16 2024-04-18T23:11:23.527 INFO:tasks.rados.rados.0.smithi142.stdout:169: oid 16 updating version 0 to 17 2024-04-18T23:11:23.527 INFO:tasks.rados.rados.0.smithi142.stdout:169: oid 16 updating version 17 to 18 2024-04-18T23:11:23.527 INFO:tasks.rados.rados.0.smithi142.stdout:169: oid 16 updating version 18 to 19 2024-04-18T23:11:23.527 INFO:tasks.rados.rados.0.smithi142.stdout:169: oid 16 version 19 is already newer than 16 2024-04-18T23:11:23.527 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 16 v 19 (ObjNum 78 snap 14 seq_num 78) dirty exists 2024-04-18T23:11:23.527 INFO:tasks.rados.rados.0.smithi142.stdout:169: left oid 16 (ObjNum 78 snap 14 seq_num 78) 2024-04-18T23:11:23.598 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T23:11:23.616 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:23.617 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:23.617 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:23.612+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:23.637 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:23.638 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:23.658 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:23.658 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:23.661 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:23.661 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:23.661 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:23.662 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:23.662 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:23.676 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:11:23.680 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:23.680 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:23.697 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:23.697 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:23.718 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:23.718 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:23.719 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:23.718+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:23.742 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:23.742 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:23.763 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:23.764 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:23.782 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/31 ECSubWrite(tid=2, reqid=client.4568.0:19, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.273032383, 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102238363, 2024-04-18T23:11:23.797 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:23.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8599999999999998e-07 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523292+0000", 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2746e-05 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523307+0000", 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5223e-05 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623426+0000", 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10011853599999999 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623593+0000", 2024-04-18T23:11:23.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016698900000000001 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623604+0000", 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1829e-05 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624978+0000", 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013738699999999999 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625487+0000", 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00050868400000000002 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 34/31 ECSubWrite(tid=0, reqid=client.4568.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.623754+0000", 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.172527086999999, 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102510163, 2024-04-18T23:11:23.802 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623754+0000", 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623754+0000", 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623755+0000", 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5000000000000002e-07 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623927+0000", 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000172802 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623957+0000", 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9016000000000001e-05 2024-04-18T23:11:23.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623977+0000", 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0886e-05 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.724391+0000", 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100413961 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.724405+0000", 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4079e-05 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726222+0000", 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001816151 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726264+0000", 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2518000000000002e-05 2024-04-18T23:11:23.806 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.172252973999999, 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10419479800000001, 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624029+0000", 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4300000000000001e-07 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624060+0000", 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1025999999999997e-05 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624083+0000", 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3323e-05 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624104+0000", 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1089000000000001e-05 2024-04-18T23:11:23.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725186+0000", 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101081146 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725195+0000", 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7829999999999993e-06 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728164+0000", 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029689249999999999 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728223+0000", 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8563000000000002e-05 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.811 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.6s4 34/31 ECSubWrite(tid=0, reqid=client.4568.0:34, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.731401+0000", 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.064879881, 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10113056600000001, 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731401+0000", 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731401+0000", 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731402+0000", 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2599999999999998e-07 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731498+0000", 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6448999999999998e-05 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731528+0000", 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9924999999999999e-05 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.731544+0000", 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6500000000000001e-05 2024-04-18T23:11:23.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.831700+0000", 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100155789 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.831710+0000", 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0006e-05 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.832492+0000", 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00078151200000000005 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.832532+0000", 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9959000000000001e-05 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 34/31 ECSubWrite(tid=2, reqid=client.4568.0:61, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:23.816 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.944827+0000", 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.851454039, 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10111951700000001, 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944827+0000", 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944827+0000", 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944828+0000", 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.257e-06 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.944993+0000", 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016476899999999999 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.945018+0000", 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222999999999999e-05 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.945033+0000", 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4859e-05 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045160+0000", 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100126494 2024-04-18T23:11:23.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045166+0000", 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3539999999999996e-06 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045905+0000", 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00073871500000000003 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:57.045947+0000", 2024-04-18T23:11:23.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1845999999999999e-05 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=56, reqid=client.4568.0:182, at_version=36'14, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.759745+0000", 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 23.036536187999999, 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060262479999999997, 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.821 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759745+0000", 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759745+0000", 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759745+0000", 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.39e-07 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759815+0000", 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9621000000000003e-05 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759824+0000", 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8089999999999999e-06 2024-04-18T23:11:23.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759832+0000", 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8909999999999995e-06 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760463+0000", 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00063172399999999996 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760470+0000", 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2199999999999997e-06 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.765740+0000", 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052702080000000002 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.765771+0000", 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1335999999999999e-05 2024-04-18T23:11:23.825 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=57, reqid=client.4568.0:183, at_version=36'15, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.759844+0000", 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 23.036437147000001, 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0059428329999999998, 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759844+0000", 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759844+0000", 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759844+0000", 2024-04-18T23:11:23.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7500000000000001e-07 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759856+0000", 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1426999999999999e-05 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759868+0000", 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2673999999999999e-05 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759875+0000", 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1380000000000001e-06 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761232+0000", 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001356583 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761237+0000", 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1590000000000001e-06 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.765776+0000", 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004538416 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.765787+0000", 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1061e-05 2024-04-18T23:11:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=63, reqid=client.4568.0:183, at_version=0'0, trim_to=0'0, roll_forward_to=36'15))", 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.068067+0000", 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.728213977999999, 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10038164300000001, 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.831 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068067+0000", 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068067+0000", 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068068+0000", 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.35e-06 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068083+0000", 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4695000000000001e-05 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068105+0000", 2024-04-18T23:11:23.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1696999999999999e-05 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068119+0000", 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4367e-05 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068210+0000", 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0946999999999995e-05 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068235+0000", 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4499e-05 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168435+0000", 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100200154 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168449+0000", 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3934000000000001e-05 2024-04-18T23:11:23.835 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.836 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.836 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.836 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.bs3 36/31 ECSubRead(tid=64, to_read={3:dd01ba5f:::smithi14230322-23:head=0,4096,0}, subchunks={3:dd01ba5f:::smithi14230322-23:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:23.836 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.068146+0000", 2024-04-18T23:11:23.836 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.728134931, 2024-04-18T23:11:23.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100284543, 2024-04-18T23:11:23.836 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068146+0000", 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068146+0000", 2024-04-18T23:11:23.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068147+0000", 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.34e-07 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068154+0000", 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9990000000000002e-06 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068165+0000", 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1968e-05 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.068176+0000", 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0018e-05 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168365+0000", 2024-04-18T23:11:23.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10018914499999999 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168431+0000", 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5979e-05 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=67, reqid=client.4568.0:189, at_version=0'0, trim_to=0'0, roll_forward_to=36'17))", 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.470594+0000", 2024-04-18T23:11:23.840 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.325687508000001, 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10183064999999999, 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470594+0000", 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470594+0000", 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470596+0000", 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0669999999999999e-06 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470621+0000", 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222999999999999e-05 2024-04-18T23:11:23.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470688+0000", 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6824999999999998e-05 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470713+0000", 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5474999999999999e-05 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.570999+0000", 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10028558899999999 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.571026+0000", 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7676000000000001e-05 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.572255+0000", 2024-04-18T23:11:23.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012286549999999999 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.572424+0000", 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016914 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 42/31 ECSubWrite(tid=85, reqid=osd.4.0:85, at_version=42'27, trim_to=0'0, roll_forward_to=40'25))", 2024-04-18T23:11:23.845 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 17.218548702, 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16837443799999999, 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577733+0000", 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6300000000000001e-07 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577758+0000", 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5099000000000001e-05 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577779+0000", 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0937999999999998e-05 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577792+0000", 2024-04-18T23:11:23.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3227999999999999e-05 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744736+0000", 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.166943963 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744769+0000", 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2629000000000002e-05 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746075+0000", 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001306338 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746107+0000", 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.188e-05 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.850 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.6s4 43/31 ECSubWrite(tid=19, reqid=osd.0.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=43'12))", 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:07.990532+0000", 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.805748714, 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10125936100000001, 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990532+0000", 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990532+0000", 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990534+0000", 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4980000000000001e-06 2024-04-18T23:11:23.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990553+0000", 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8848999999999999e-05 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990575+0000", 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2339000000000001e-05 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990587+0000", 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2340999999999999e-05 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990737+0000", 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015011700000000001 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990774+0000", 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6745999999999997e-05 2024-04-18T23:11:23.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:08.091745+0000", 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100970571 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:08.091792+0000", 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6900000000000002e-05 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.bs3 46/31 ECSubRead(tid=108, to_read={3:d447fef0:::smithi14230322-49:head=0,204800,0}, subchunks={3:d447fef0:::smithi14230322-49:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:10.710122+0000", 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.086158851, 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053464286, 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.710122+0000", 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.710122+0000", 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.710123+0000", 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0890000000000001e-06 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.710134+0000", 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0862e-05 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.710157+0000", 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3326999999999999e-05 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.710173+0000", 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5634999999999999e-05 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.763398+0000", 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.053225384000000001 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.763586+0000", 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018798900000000001 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.859 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 46/31 ECSubWrite(tid=113, reqid=client.4568.0:326, at_version=0'0, trim_to=0'0, roll_forward_to=46'31))", 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:11.935715+0000", 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.860566511, 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102077656, 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935715+0000", 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935715+0000", 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935716+0000", 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.497e-06 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935731+0000", 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4958e-05 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935752+0000", 2024-04-18T23:11:23.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0815999999999999e-05 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935765+0000", 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3481e-05 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.036081+0000", 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10031585699999999 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.036108+0000", 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7282999999999999e-05 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.037695+0000", 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001586524 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.037792+0000", 2024-04-18T23:11:23.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7239999999999997e-05 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 47/31 ECSubWrite(tid=26, reqid=client.4568.0:333, at_version=47'16, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.06439694, 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201860068, 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.865 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6299999999999999e-07 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732265+0000", 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038060399999999999 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732281+0000", 2024-04-18T23:11:23.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6362e-05 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732292+0000", 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0817e-05 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.832471+0000", 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100179254 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.832486+0000", 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4911e-05 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933708+0000", 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101221567 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933744+0000", 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6390000000000002e-05 2024-04-18T23:11:23.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 47/31 ECSubWrite(tid=27, reqid=client.4568.0:344, at_version=47'18, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.061749605999999, 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19924303900000001, 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.870 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.871 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734532+0000", 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734555+0000", 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3258999999999999e-05 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734568+0000", 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2568999999999999e-05 2024-04-18T23:11:23.872 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734575+0000", 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6279999999999999e-06 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.833597+0000", 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099021993000000003 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.873 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.833605+0000", 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0620000000000001e-06 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933755+0000", 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10015038 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933774+0000", 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9474999999999999e-05 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.874 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 47/31 ECSubWrite(tid=119, reqid=client.4568.0:336, at_version=0'0, trim_to=0'0, roll_forward_to=47'36))", 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:13.269515+0000", 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.526766444, 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10301724499999999, 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.875 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269515+0000", 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269515+0000", 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269517+0000", 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6529999999999998e-06 2024-04-18T23:11:23.876 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269550+0000", 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2605999999999999e-05 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269587+0000", 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7452e-05 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.877 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.369688+0000", 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10010050400000001 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.369783+0000", 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5655000000000005e-05 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.369795+0000", 2024-04-18T23:11:23.878 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1705999999999999e-05 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.372486+0000", 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026909960000000002 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.372532+0000", 2024-04-18T23:11:23.879 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5673000000000001e-05 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 47/31 ECSubWrite(tid=21, reqid=client.4568.0:358, at_version=47'6, trim_to=0'0, roll_forward_to=34'4))", 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:13.562870+0000", 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.233410592, 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0052486039999999996, 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.880 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.562870+0000", 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.562870+0000", 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.881 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.562872+0000", 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7880000000000001e-06 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.565855+0000", 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029832000000000001 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.565882+0000", 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6315000000000001e-05 2024-04-18T23:11:23.882 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.565897+0000", 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5727999999999999e-05 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.566049+0000", 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000151836 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.883 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.566064+0000", 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4628e-05 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.568000+0000", 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001935588 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.568119+0000", 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000119521 2024-04-18T23:11:23.884 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 57/31 ECSubWrite(tid=168, reqid=osd.4.0:168, at_version=57'60, trim_to=0'0, roll_forward_to=55'58))", 2024-04-18T23:11:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.629337+0000", 2024-04-18T23:11:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.166943829, 2024-04-18T23:11:23.885 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.1034201, 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629337+0000", 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.886 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629337+0000", 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629337+0000", 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9799999999999999e-07 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629364+0000", 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6950000000000001e-05 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.887 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629394+0000", 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9232000000000001e-05 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629406+0000", 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1987000000000001e-05 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.730744+0000", 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101338254 2024-04-18T23:11:23.888 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.730777+0000", 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3071999999999998e-05 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.732693+0000", 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019157060000000001 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.889 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.732757+0000", 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4601e-05 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 57/31 ECSubWrite(tid=169, reqid=osd.4.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=57'60))", 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.733631+0000", 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.0626500329999999, 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101563061, 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:23.890 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733631+0000", 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733631+0000", 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.891 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733632+0000", 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4479999999999999e-06 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733648+0000", 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5964000000000002e-05 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733672+0000", 2024-04-18T23:11:23.892 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3731e-05 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733689+0000", 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7326999999999999e-05 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733840+0000", 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015033700000000001 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.893 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733866+0000", 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6163e-05 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.835077+0000", 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101210974 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.835194+0000", 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117117 2024-04-18T23:11:23.894 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:23.895 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:23.895 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:23.897 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:11:23.897 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:23.898 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:23.898 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:23.898 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:23.899 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:23.899 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:23.899 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:23.900 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:23.900 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:23.894+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:23.901 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:23.901 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:23.922 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:23.922 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:23.922 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENXIO: problem getting command descriptions from osd.15 2024-04-18T23:11:23.926 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T23:11:23.926 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T23:11:23.997 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:23.996+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:24.030 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:24.046 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:11:24.098 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:24.100+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:24.199 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:24.198+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:24.215 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:24.215 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:24.229 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:11:24.301 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:24.298+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:24.351 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:24.352 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:24.367 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:11:24.410 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:24.412+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:24.486 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:24.487 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:24.500 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:11:24.502 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:24.498+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:24.535 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-04-18T23:11:24.536 INFO:tasks.rados.rados.0.smithi142.stdout:169: done (1 left) 2024-04-18T23:11:24.536 INFO:tasks.rados.rados.0.smithi142.stdout:176: done (0 left) 2024-04-18T23:11:24.536 INFO:tasks.rados.rados.0.smithi142.stdout:177: read oid 46 snap -1 2024-04-18T23:11:24.536 INFO:tasks.rados.rados.0.smithi142.stdout:177: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-18T23:11:24.536 INFO:tasks.rados.rados.0.smithi142.stdout:178: read oid 7 snap 5 2024-04-18T23:11:24.536 INFO:tasks.rados.rados.0.smithi142.stdout:178: expect (ObjNum 6 snap 0 seq_num 6) 2024-04-18T23:11:24.536 INFO:tasks.rados.rados.0.smithi142.stdout:179: snap_create 2024-04-18T23:11:24.603 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:24.600+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:24.658 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:24.658 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:24.672 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:11:24.704 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:24.702+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:24.792 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:24.792 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:24.804 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:24.802+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:24.806 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:11:24.905 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:24.904+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:24.930 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:24.930 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:24.945 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:11:25.006 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:25.002+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:25.068 INFO:tasks.rados.rados.0.smithi142.stdout:178: expect (ObjNum 6 snap 0 seq_num 6) 2024-04-18T23:11:25.107 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:25.102+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:25.108 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:25.123 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:11:25.145 INFO:tasks.rados.rados.0.smithi142.stdout:177: expect (ObjNum 45 snap 0 seq_num 45) 2024-04-18T23:11:25.208 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:25.206+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:25.244 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:25.244 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:25.258 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:11:25.309 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:25.310+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:25.382 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:25.382 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:25.396 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:11:25.410 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:25.408+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:25.511 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:25.510+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:25.512 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:25.513 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:25.525 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:11:25.542 INFO:tasks.rados.rados.0.smithi142.stdout:177: done (2 left) 2024-04-18T23:11:25.542 INFO:tasks.rados.rados.0.smithi142.stdout:178: done (1 left) 2024-04-18T23:11:25.542 INFO:tasks.rados.rados.0.smithi142.stdout:179: done (0 left) 2024-04-18T23:11:25.542 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 13 current snap is 16 2024-04-18T23:11:25.542 INFO:tasks.rados.rados.0.smithi142.stdout:180: seq_num 80 ranges {212992=589824,802816=573440,1376256=196608} 2024-04-18T23:11:25.555 INFO:tasks.rados.rados.0.smithi142.stdout:180: writing smithi14230322-13 from 212992 to 802816 tid 1 2024-04-18T23:11:25.564 INFO:tasks.rados.rados.0.smithi142.stdout:180: writing smithi14230322-13 from 802816 to 1376256 tid 2 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:180: writing smithi14230322-13 from 1376256 to 1572864 tid 3 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:181: copy_from oid 15 from oid 5 current snap is 16 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:182: rollback oid 19 current snap is 16 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 19 to 10 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:183: copy_from oid 21 from oid 24 current snap is 16 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:184: copy_from oid 3 from oid 7 current snap is 16 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:185: read oid 33 snap -1 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:185: expect (ObjNum 57 snap 4 seq_num 57) 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 16 current snap is 16 2024-04-18T23:11:25.568 INFO:tasks.rados.rados.0.smithi142.stdout:186: seq_num 81 ranges {2555904=606208,3162112=720896,3883008=49152} 2024-04-18T23:11:25.611 INFO:tasks.rados.rados.0.smithi142.stdout:186: writing smithi14230322-16 from 2555904 to 3162112 tid 1 2024-04-18T23:11:25.612 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:25.610+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:25.618 INFO:tasks.rados.rados.0.smithi142.stdout:186: writing smithi14230322-16 from 3162112 to 3883008 tid 2 2024-04-18T23:11:25.618 INFO:tasks.rados.rados.0.smithi142.stdout:186: writing smithi14230322-16 from 3883008 to 3932160 tid 3 2024-04-18T23:11:25.618 INFO:tasks.rados.rados.0.smithi142.stdout:187: rmattr oid 34 current snap is 16 2024-04-18T23:11:25.619 INFO:tasks.rados.rados.0.smithi142.stdout:187: done (7 left) 2024-04-18T23:11:25.619 INFO:tasks.rados.rados.0.smithi142.stdout:188: snap_remove snap 5 2024-04-18T23:11:25.647 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:25.647 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:25.662 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:11:25.713 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:25.708+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:25.793 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:25.793 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:25.809 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:11:25.813 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:25.812+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:25.914 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:25.910+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:25.927 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:25.928 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:25.928 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:25.944 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:11:26.015 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:26.010+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:26.063 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:26.063 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:26.077 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:11:26.084 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:26.084 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:26.106 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:26.106 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:26.117 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:26.116+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:26.124 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:26.146 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:26.146 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:26.163 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:26.164 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:26.185 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:26.185 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:26.201 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:26.201 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:26.206 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:26.207 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:26.217 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:11:26.218 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:26.214+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:26.227 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:26.227 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:26.245 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:26.245 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:26.266 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:26.266 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:26.270 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [15] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11] 2024-04-18T23:11:26.270 INFO:tasks.thrashosds.thrasher:Reviving osd 15 2024-04-18T23:11:26.270 INFO:tasks.ceph.osd.15:Restarting daemon 2024-04-18T23:11:26.271 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 15 2024-04-18T23:11:26.273 INFO:tasks.ceph.osd.15:Started 2024-04-18T23:11:26.273 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:11:26.286 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:26.287 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:26.307 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:26.307 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:26.318 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:26.316+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:26.325 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:26.346 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:26.347 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:26.368 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:26.369 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:26.369 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENXIO: problem getting command descriptions from osd.15 2024-04-18T23:11:26.372 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T23:11:26.373 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T23:11:26.379 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:26.379 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:26.380 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:26.380 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:26.380 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:26.394 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:11:26.400 INFO:teuthology.orchestra.run.smithi142.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:0 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:1 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:2 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:abort 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:assert 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:config diff 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:config diff get 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:config get 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:config help [] 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:config set ... 2024-04-18T23:11:26.401 INFO:teuthology.orchestra.run.smithi142.stderr:admin_socket: invalid command 2024-04-18T23:11:26.403 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:26.403 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-15, ['dump_ops_in_flight'] 2024-04-18T23:11:26.419 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:26.419+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:26.513 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:26.513 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:26.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=158, reqid=client.4568.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.885518506, 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203058724, 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:11:26.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627730+0000", 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.606e-06 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627841+0000", 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110692 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627889+0000", 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8211000000000003e-05 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627919+0000", 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9961999999999999e-05 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629700+0000", 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001780963 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629730+0000", 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0082000000000001e-05 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830736+0000", 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201005607 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830787+0000", 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1601000000000003e-05 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=159, reqid=client.4568.0:25, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.885244693000001, 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20284285199999999, 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628004+0000", 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0270000000000001e-06 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628137+0000", 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013336000000000001 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628197+0000", 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0504000000000002e-05 2024-04-18T23:11:26.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628221+0000", 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3037999999999999e-05 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728171+0000", 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099950707999999999 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728190+0000", 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8709e-05 2024-04-18T23:11:26.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830795+0000", 2024-04-18T23:11:26.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102605138 2024-04-18T23:11:26.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830845+0000", 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0368000000000003e-05 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=160, reqid=client.4568.0:26, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.884989972, 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20264686400000001, 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.523 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628258+0000", 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628292+0000", 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4054999999999998e-05 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628317+0000", 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5035999999999999e-05 2024-04-18T23:11:26.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628338+0000", 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1058999999999999e-05 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728828+0000", 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100489521 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728846+0000", 2024-04-18T23:11:26.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8419000000000001e-05 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830850+0000", 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10200397 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830904+0000", 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3844000000000002e-05 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=161, reqid=client.4568.0:31, at_version=34'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.884855928, 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202594633, 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628392+0000", 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6000000000000004e-07 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628541+0000", 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149386 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628572+0000", 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0975000000000001e-05 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628594+0000", 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1608999999999999e-05 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829299+0000", 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20070527499999999 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829314+0000", 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4718999999999999e-05 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830915+0000", 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001601412 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830986+0000", 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0697000000000004e-05 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.531 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=162, reqid=client.4568.0:32, at_version=34'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.635797+0000", 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.877450173, 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.195800061, 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635797+0000", 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635797+0000", 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635799+0000", 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9039999999999999e-06 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635849+0000", 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0281000000000002e-05 2024-04-18T23:11:26.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635897+0000", 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7416999999999997e-05 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635923+0000", 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6486e-05 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830004+0000", 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19408114400000001 2024-04-18T23:11:26.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830019+0000", 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4533e-05 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.831518+0000", 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014992300000000001 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.831597+0000", 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9066000000000005e-05 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.535 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 36/31 ECSubWrite(tid=54, reqid=client.4568.0:188, at_version=36'19, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.762269+0000", 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.750978522, 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103903021, 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762269+0000", 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762269+0000", 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762269+0000", 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5400000000000001e-07 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762379+0000", 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110065 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762407+0000", 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7515e-05 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762427+0000", 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0392999999999999e-05 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762564+0000", 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013708399999999999 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.762583+0000", 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8868999999999998e-05 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866094+0000", 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103510981 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866172+0000", 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7459999999999994e-05 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 36/31 ECSubWrite(tid=58, reqid=client.4568.0:196, at_version=36'21, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.763024+0000", 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.750222837999999, 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103273062, 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.540 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763024+0000", 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763024+0000", 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763025+0000", 2024-04-18T23:11:26.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2e-07 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763067+0000", 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2135000000000002e-05 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763078+0000", 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1362000000000001e-05 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763088+0000", 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1900000000000001e-06 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763717+0000", 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000629179 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763741+0000", 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4572000000000001e-05 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866222+0000", 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102480577 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866297+0000", 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5626999999999999e-05 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 36/31 ECSubWrite(tid=59, reqid=client.4568.0:197, at_version=36'22, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.763568+0000", 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.749679448999998, 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103433918, 2024-04-18T23:11:26.544 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763568+0000", 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763568+0000", 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763569+0000", 2024-04-18T23:11:26.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.06e-06 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763596+0000", 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7013e-05 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763627+0000", 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1531e-05 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.763639+0000", 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.206e-05 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.865607+0000", 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101967455 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.865622+0000", 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4973e-05 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866932+0000", 2024-04-18T23:11:26.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013096029999999999 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.867002+0000", 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0222999999999995e-05 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 36/31 ECSubWrite(tid=60, reqid=client.4568.0:198, at_version=36'23, trim_to=0'0, roll_forward_to=35'17))", 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.764316+0000", 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.748930842, 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103564976, 2024-04-18T23:11:26.548 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764316+0000", 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764316+0000", 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.549 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764318+0000", 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4169999999999999e-06 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764357+0000", 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9499000000000002e-05 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764418+0000", 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0603999999999998e-05 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.550 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.764439+0000", 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1239000000000001e-05 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866401+0000", 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101961882 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.866431+0000", 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9578999999999999e-05 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.551 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.867805+0000", 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013745059999999999 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.867881+0000", 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6249999999999997e-05 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 36/31 ECSubWrite(tid=9, reqid=client.4568.0:176, at_version=36'8, trim_to=0'0, roll_forward_to=34'4))", 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.956436+0000", 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.556810887000001, 2024-04-18T23:11:26.552 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10256767999999999, 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956436+0000", 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956436+0000", 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.553 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956438+0000", 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956564+0000", 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000126317 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956589+0000", 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5148999999999999e-05 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.554 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.956613+0000", 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3731999999999999e-05 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056863+0000", 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10025031700000001 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.056887+0000", 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3552000000000001e-05 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.058940+0000", 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020530589999999999 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.059004+0000", 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4073999999999997e-05 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.es5 42/31 ECSubWrite(tid=25, reqid=osd.2.0:24, at_version=0'0, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.799564+0000", 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.713683729, 2024-04-18T23:11:26.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18457921799999999, 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799564+0000", 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799564+0000", 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799565+0000", 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799594+0000", 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8221999999999998e-05 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799640+0000", 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6749000000000003e-05 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.799665+0000", 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4712e-05 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.883580+0000", 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.083914739000000002 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.883592+0000", 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2595000000000001e-05 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.984082+0000", 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100489984 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.984143+0000", 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0404e-05 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 42/31 ECSubWrite(tid=89, reqid=client.4568.0:279, at_version=42'41, trim_to=0'0, roll_forward_to=42'39))", 2024-04-18T23:11:26.560 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.094462+0000", 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 19.418785593999999, 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10436654099999999, 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.094462+0000", 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.094462+0000", 2024-04-18T23:11:26.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.094464+0000", 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0200000000000001e-06 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.095463+0000", 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0009991100000000001 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.095494+0000", 2024-04-18T23:11:26.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0691999999999997e-05 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.195636+0000", 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100142097 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.195800+0000", 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000164057 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.195815+0000", 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4972e-05 2024-04-18T23:11:26.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.198757+0000", 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029427590000000001 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.198828+0000", 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0834000000000007e-05 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s5 42/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=42'12))", 2024-04-18T23:11:26.564 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.669347+0000", 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.843900442999999, 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20082834299999999, 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669347+0000", 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669347+0000", 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669348+0000", 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7400000000000002e-07 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669371+0000", 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3332e-05 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669458+0000", 2024-04-18T23:11:26.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6646999999999999e-05 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669484+0000", 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6706000000000001e-05 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.769505+0000", 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100021191 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.567 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.769528+0000", 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2132000000000001e-05 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.870118+0000", 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100590156 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.870175+0000", 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7404999999999997e-05 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s6 43/31 ECSubWrite(tid=36, reqid=client.4568.0:291, at_version=43'23, trim_to=0'0, roll_forward_to=42'17))", 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.792105+0000", 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.721142016000002, 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103170358, 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792105+0000", 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792105+0000", 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792112+0000", 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8619999999999997e-06 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792145+0000", 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2571999999999999e-05 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.792172+0000", 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7253000000000001e-05 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.892267+0000", 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100095301 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.892409+0000", 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141695 2024-04-18T23:11:26.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.892441+0000", 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2016e-05 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.895187+0000", 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027456820000000002 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.895276+0000", 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8976999999999999e-05 2024-04-18T23:11:26.572 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 46/31 ECSubWrite(tid=20, reqid=client.4568.0:331, at_version=46'14, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.432076+0000", 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.081171238, 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20163852800000001, 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432076+0000", 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432076+0000", 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432077+0000", 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1399999999999997e-07 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432118+0000", 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1155999999999997e-05 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432134+0000", 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6198e-05 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432147+0000", 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3385e-05 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.632376+0000", 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200228505 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.632385+0000", 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2879999999999998e-06 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.633631+0000", 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001246268 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.633715+0000", 2024-04-18T23:11:26.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3114000000000004e-05 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 46/31 ECSubWrite(tid=21, reqid=client.4568.0:331, at_version=0'0, trim_to=0'0, roll_forward_to=46'14))", 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.635045+0000", 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.878202673000001, 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103353785, 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.635045+0000", 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.635045+0000", 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.635047+0000", 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.103e-06 2024-04-18T23:11:26.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.635106+0000", 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9444999999999998e-05 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.635143+0000", 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6412000000000003e-05 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.735236+0000", 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009332999999999 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.735371+0000", 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013544499999999999 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.735387+0000", 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5451000000000002e-05 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.738296+0000", 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002908842 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.738398+0000", 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010275700000000001 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs4 47/31 ECSubWrite(tid=114, reqid=osd.4.0:114, at_version=47'33, trim_to=0'0, roll_forward_to=46'31))", 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.630014+0000", 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.883233586999999, 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20008867999999999, 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630014+0000", 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630014+0000", 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630014+0000", 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6600000000000002e-07 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630056+0000", 2024-04-18T23:11:26.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1798999999999997e-05 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630088+0000", 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2441999999999999e-05 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.730185+0000", 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100096952 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828149+0000", 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.097963921999999995 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828173+0000", 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3941999999999999e-05 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830006+0000", 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001833243 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830102+0000", 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6014000000000006e-05 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.584 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs4 47/31 ECSubWrite(tid=115, reqid=client.4568.0:334, at_version=47'34, trim_to=0'0, roll_forward_to=46'31))", 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.833782+0000", 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.67946491, 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105592888, 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.833782+0000", 2024-04-18T23:11:26.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.833782+0000", 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.833784+0000", 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5430000000000001e-06 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.834410+0000", 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00062601900000000001 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.834440+0000", 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9621999999999999e-05 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.934544+0000", 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100104295 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.934705+0000", 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016130499999999999 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.934721+0000", 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5696e-05 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.939303+0000", 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0045824469999999999 2024-04-18T23:11:26.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.939375+0000", 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1960999999999998e-05 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.es5 47/31 ECSubWrite(tid=28, reqid=client.4568.0:344, at_version=0'0, trim_to=0'0, roll_forward_to=47'18))", 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.937826+0000", 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.575420773999999, 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102106899, 2024-04-18T23:11:26.589 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937826+0000", 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937826+0000", 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937828+0000", 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7430000000000001e-06 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937891+0000", 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2261000000000006e-05 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937955+0000", 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4074999999999999e-05 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.038072+0000", 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10011761199999999 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.038225+0000", 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015305200000000001 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.038252+0000", 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6288999999999999e-05 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.039846+0000", 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015940559999999999 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.039933+0000", 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7811000000000003e-05 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 56/31 ECSubWrite(tid=164, reqid=client.4568.0:431, at_version=56'73, trim_to=0'0, roll_forward_to=55'71))", 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9677353870000003, 2024-04-18T23:11:26.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20358216400000001, 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7300000000000002e-07 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545582+0000", 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9449000000000005e-05 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545607+0000", 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5741999999999998e-05 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:26.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545625+0000", 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7503000000000002e-05 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.645851+0000", 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100225733 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.645870+0000", 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8972999999999999e-05 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749022+0000", 2024-04-18T23:11:26.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103152813 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749094+0000", 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1577999999999998e-05 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:26.597 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:26.598 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T23:11:26.599 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:26.599+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:26.600 INFO:tasks.rados.rados.0.smithi142.stdout:188: done (7 left) 2024-04-18T23:11:26.600 INFO:tasks.rados.rados.0.smithi142.stdout:189: read oid 39 snap -1 2024-04-18T23:11:26.600 INFO:tasks.rados.rados.0.smithi142.stdout:189: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-18T23:11:26.600 INFO:tasks.rados.rados.0.smithi142.stdout:190: setattr oid 40 current snap is 16 2024-04-18T23:11:26.600 INFO:tasks.rados.rados.0.smithi142.stdout:182: finishing rollback tid 1 to smithi14230322-19 2024-04-18T23:11:26.600 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 19 v 22 (ObjNum 4113 snap 0 seq_num 1387512576) dirty dne 2024-04-18T23:11:26.600 INFO:tasks.rados.rados.0.smithi142.stdout:186: finishing write tid 1 to smithi14230322-16 2024-04-18T23:11:26.600 INFO:tasks.rados.rados.0.smithi142.stdout:186: finishing write tid 2 to smithi14230322-16 2024-04-18T23:11:26.600 INFO:tasks.rados.rados.0.smithi142.stdout:186: finishing write tid 3 to smithi14230322-16 2024-04-18T23:11:26.601 INFO:tasks.rados.rados.0.smithi142.stdout:184: finishing copy_from racing read to smithi14230322-3 2024-04-18T23:11:26.601 INFO:tasks.rados.rados.0.smithi142.stdout:181: finishing copy_from racing read to smithi14230322-15 2024-04-18T23:11:26.601 INFO:tasks.rados.rados.0.smithi142.stdout:186: finishing write tid 4 to smithi14230322-16 2024-04-18T23:11:26.601 INFO:tasks.rados.rados.0.smithi142.stdout:184: finishing copy_from to smithi14230322-3 2024-04-18T23:11:26.601 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 3 v 21 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-18T23:11:26.601 INFO:tasks.rados.rados.0.smithi142.stdout:185: expect (ObjNum 57 snap 4 seq_num 57) 2024-04-18T23:11:26.601 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:11:26.634 INFO:tasks.rados.rados.0.smithi142.stdout:183: finishing copy_from racing read to smithi14230322-21 2024-04-18T23:11:26.634 INFO:tasks.rados.rados.0.smithi142.stdout:180: finishing write tid 1 to smithi14230322-13 2024-04-18T23:11:26.634 INFO:tasks.rados.rados.0.smithi142.stdout:181: finishing copy_from to smithi14230322-15 2024-04-18T23:11:26.634 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 15 v 2 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-18T23:11:26.634 INFO:tasks.rados.rados.0.smithi142.stdout:180: finishing write tid 2 to smithi14230322-13 2024-04-18T23:11:26.634 INFO:tasks.rados.rados.0.smithi142.stdout:180: finishing write tid 3 to smithi14230322-13 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:180: finishing write tid 4 to smithi14230322-13 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:183: finishing copy_from to smithi14230322-21 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 21 v 4 (ObjNum 28 snap 0 seq_num 28) dirty exists 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:180: finishing write tid 5 to smithi14230322-13 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:180: oid 13 updating version 0 to 26 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:180: oid 13 updating version 26 to 27 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:180: oid 13 version 27 is already newer than 25 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:180: oid 13 version 27 is already newer than 24 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 13 v 27 (ObjNum 80 snap 16 seq_num 80) dirty exists 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:180: left oid 13 (ObjNum 80 snap 16 seq_num 80) 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:186: finishing write tid 5 to smithi14230322-16 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:186: oid 16 updating version 0 to 24 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:186: oid 16 version 24 is already newer than 22 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:186: oid 16 version 24 is already newer than 23 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:186: oid 16 version 24 is already newer than 21 2024-04-18T23:11:26.635 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 16 v 24 (ObjNum 81 snap 16 seq_num 81) dirty exists 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:186: left oid 16 (ObjNum 81 snap 16 seq_num 81) 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:180: done (8 left) 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:181: done (7 left) 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:182: done (6 left) 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:183: done (5 left) 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:184: done (4 left) 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:185: done (3 left) 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:186: done (2 left) 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:191: copy_from oid 25 from oid 36 current snap is 16 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:192: copy_from oid 46 from oid 3 current snap is 16 2024-04-18T23:11:26.636 INFO:tasks.rados.rados.0.smithi142.stdout:193: snap_create 2024-04-18T23:11:26.700 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:26.694+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:26.733 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:26.733 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:26.733 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:11:26.733 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:26.749 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:11:26.800 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:26.799+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:26.842 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 40 v 40 (ObjNum 129 snap 0 seq_num 3924337248) dirty exists 2024-04-18T23:11:26.845 INFO:tasks.rados.rados.0.smithi142.stdout:192: finishing copy_from racing read to smithi14230322-46 2024-04-18T23:11:26.872 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:26.872 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:26.872 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:26.872 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:26.872 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:26.886 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:11:26.907 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:26.904+0000 7f742b85f7c0 -1 Falling back to public interface 2024-04-18T23:11:26.930 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:26.930+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:26.941 INFO:tasks.rados.rados.0.smithi142.stdout:189: expect (ObjNum 38 snap 0 seq_num 38) 2024-04-18T23:11:26.997 INFO:tasks.rados.rados.0.smithi142.stdout:191: finishing copy_from racing read to smithi14230322-25 2024-04-18T23:11:27.001 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:26.999+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:27.010 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:27.010 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:27.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:27.010 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 34/31 ECSubWrite(tid=0, reqid=client.4568.0:7, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.528298+0000", 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.482029709999999, 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003543147, 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528298+0000", 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528298+0000", 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528299+0000", 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9600000000000008e-07 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528473+0000", 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017360800000000001 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528507+0000", 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3971999999999999e-05 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.528538+0000", 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1684999999999999e-05 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.529242+0000", 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00070414900000000005 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.529255+0000", 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2850000000000001e-05 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.531741+0000", 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0024855620000000002 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.531841+0000", 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100325 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.014 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 34/31 ECSubWrite(tid=0, reqid=client.4568.0:28, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.535047+0000", 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.475280602000002, 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099095934999999996, 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535047+0000", 2024-04-18T23:11:27.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535047+0000", 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535048+0000", 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.96e-07 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535249+0000", 2024-04-18T23:11:27.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020098699999999999 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535280+0000", 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1301999999999999e-05 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.535304+0000", 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3918999999999999e-05 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.632426+0000", 2024-04-18T23:11:27.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.097122690999999997 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.632441+0000", 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4273e-05 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.634033+0000", 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001592729 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.634143+0000", 2024-04-18T23:11:27.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000109638 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:29, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.636644+0000", 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.373684086000001, 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10234691999999999, 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636644+0000", 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636644+0000", 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636646+0000", 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.103e-06 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636689+0000", 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3464999999999998e-05 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636729+0000", 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0005000000000001e-05 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636763+0000", 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3417999999999998e-05 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.737095+0000", 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100332765 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.737122+0000", 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6355000000000001e-05 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.738921+0000", 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017988730000000001 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.738991+0000", 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9936000000000003e-05 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.1s4 34/31 ECSubWrite(tid=10, reqid=client.4568.0:2, at_version=0'0, trim_to=0'0, roll_forward_to=33'2))", 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.825267+0000", 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 30.185060706000002, 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10131675399999999, 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.023 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825267+0000", 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825267+0000", 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825269+0000", 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.844e-06 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825298+0000", 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9065e-05 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.825331+0000", 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3062000000000003e-05 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.925491+0000", 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100160233 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.925692+0000", 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000201263 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.925708+0000", 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5913000000000002e-05 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.926517+0000", 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00080818100000000005 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.926584+0000", 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7193000000000005e-05 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 34/31 ECSubWrite(tid=3, reqid=client.4568.0:62, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:57.047096+0000", 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.963231541999999, 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035305250000000001, 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.027 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047096+0000", 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047096+0000", 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047097+0000", 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2899999999999999e-06 2024-04-18T23:11:27.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047140+0000", 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2166999999999998e-05 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047166+0000", 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5939e-05 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.047187+0000", 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1866e-05 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.029 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.048317+0000", 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011297830000000001 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.048344+0000", 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6911999999999999e-05 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.050579+0000", 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002234556 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:57.050627+0000", 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8012e-05 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 36/31 ECSubRead(tid=22, to_read={3:b210783d:::smithi14230322-30:head=0,122880,0}, subchunks={3:b210783d:::smithi14230322-30:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.545989+0000", 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.464338502, 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10058726900000001, 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.545989+0000", 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.545989+0000", 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.545990+0000", 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4299999999999999e-07 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546004+0000", 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4523e-05 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546036+0000", 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1429e-05 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546068+0000", 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2654999999999998e-05 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646370+0000", 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100302105 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646577+0000", 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000206214 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 36/31 ECSubRead(tid=23, to_read={3:b210783d:::smithi14230322-30:head=0,122880,0}, subchunks={3:b210783d:::smithi14230322-30:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:27.034 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.546648+0000", 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.463680160999999, 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100272213, 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546648+0000", 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546648+0000", 2024-04-18T23:11:27.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546649+0000", 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6139999999999999e-06 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546665+0000", 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6016e-05 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546700+0000", 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4857999999999997e-05 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.546730+0000", 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0232000000000001e-05 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646687+0000", 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099956749999999997 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.646920+0000", 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023274299999999999 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 36/31 ECSubRead(tid=24, to_read={3:b210783d:::smithi14230322-30:head=0,122880,0}, subchunks={3:b210783d:::smithi14230322-30:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.548897+0000", 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.461430619000001, 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098321620999999998, 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548897+0000", 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548897+0000", 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548899+0000", 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.607e-06 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548915+0000", 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5985999999999999e-05 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548950+0000", 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4972000000000003e-05 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.548999+0000", 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9740999999999997e-05 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.647040+0000", 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098040085999999999 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.647219+0000", 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017922899999999999 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=25, reqid=client.4568.0:185, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.360244438999999, 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103575165, 2024-04-18T23:11:27.041 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650085+0000", 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8929999999999999e-06 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650164+0000", 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8852999999999999e-05 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650193+0000", 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9352e-05 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750332+0000", 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100138779 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750511+0000", 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000178679 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750538+0000", 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328999999999999e-05 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753559+0000", 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030212080000000001 2024-04-18T23:11:27.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753658+0000", 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9072000000000003e-05 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=26, reqid=client.4568.0:186, at_version=36'14, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.750398+0000", 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.259929887999998, 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0038214529999999998, 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.045 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750398+0000", 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750398+0000", 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750399+0000", 2024-04-18T23:11:27.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0899999999999999e-06 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750442+0000", 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3115000000000003e-05 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750469+0000", 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6579000000000001e-05 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.047 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750490+0000", 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1075000000000001e-05 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752513+0000", 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020229050000000002 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.752538+0000", 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5868999999999999e-05 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.754153+0000", 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016146260000000001 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.754219+0000", 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6193999999999997e-05 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.049 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=29, reqid=client.4568.0:202, at_version=36'17, trim_to=0'0, roll_forward_to=36'15))", 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:01.073962+0000", 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 25.93636549, 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10286168, 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073962+0000", 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073962+0000", 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073964+0000", 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6300000000000001e-06 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.074019+0000", 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5228e-05 2024-04-18T23:11:27.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.074049+0000", 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0009e-05 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.174167+0000", 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100118289 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.174350+0000", 2024-04-18T23:11:27.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018256199999999999 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.174375+0000", 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5023000000000001e-05 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.176733+0000", 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023585030000000001 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.053 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.176824+0000", 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0435999999999999e-05 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 46/31 ECSubRead(tid=38, to_read={3:ba4438ef:::smithi14230322-48:head=0,8192,0}, subchunks={3:ba4438ef:::smithi14230322-48:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:10.714783+0000", 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 16.295544830000001, 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099622317000000002, 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.054 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714783+0000", 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714783+0000", 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714784+0000", 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5799999999999999e-07 2024-04-18T23:11:27.055 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714790+0000", 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.976e-06 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714815+0000", 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5474e-05 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.056 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.714829+0000", 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3957999999999999e-05 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.814241+0000", 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099411553999999999 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:10.814405+0000", 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000164697 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.057 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 47/31 ECSubWrite(tid=79, reqid=client.4568.0:341, at_version=47'31, trim_to=0'0, roll_forward_to=45'29))", 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.830588+0000", 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.179739961999999, 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.096432239000000003, 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830588+0000", 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.058 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830588+0000", 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830589+0000", 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.477e-06 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830643+0000", 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4055000000000003e-05 2024-04-18T23:11:27.059 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830683+0000", 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9901999999999998e-05 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830712+0000", 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8901999999999999e-05 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830945+0000", 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00023285600000000001 2024-04-18T23:11:27.060 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830979+0000", 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4529000000000001e-05 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.926957+0000", 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.095977442999999996 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.927020+0000", 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3075000000000002e-05 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.061 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/31 ECSubWrite(tid=48, reqid=osd.7.0:48, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.846919+0000", 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.163408975999999, 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10250812300000001, 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.846919+0000", 2024-04-18T23:11:27.062 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.846919+0000", 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.846920+0000", 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5340000000000001e-06 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.846994+0000", 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3783999999999997e-05 2024-04-18T23:11:27.063 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.847023+0000", 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8535e-05 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.847058+0000", 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5101999999999997e-05 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947290+0000", 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10023225099999999 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.064 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947320+0000", 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0395999999999998e-05 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.949355+0000", 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020347059999999998 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.949427+0000", 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1815000000000003e-05 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.065 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.fs5 51/31 ECSubWrite(tid=23, reqid=client.4568.0:384, at_version=51'4, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:16.774270+0000", 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.236058177, 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0042551409999999996, 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.774270+0000", 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.066 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.774270+0000", 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.774270+0000", 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3e-07 2024-04-18T23:11:27.067 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.774599+0000", 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00032840700000000002 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.774629+0000", 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0298999999999999e-05 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.774648+0000", 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9012999999999999e-05 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.068 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.774833+0000", 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000185267 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.774865+0000", 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1683000000000001e-05 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.778399+0000", 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035341819999999999 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.069 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:16.778525+0000", 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012566000000000001 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 56/31 ECSubWrite(tid=64, reqid=client.4568.0:426, at_version=56'42, trim_to=0'0, roll_forward_to=54'40))", 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.955591+0000", 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0547368060000002, 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0050606749999999997, 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.070 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.955591+0000", 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.955591+0000", 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.071 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.955593+0000", 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9800000000000001e-06 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.956338+0000", 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00074523700000000005 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.956371+0000", 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2398999999999999e-05 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.072 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.956387+0000", 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6920000000000001e-05 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.956537+0000", 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149066 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.956556+0000", 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9446e-05 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.073 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.960560+0000", 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040040359999999999 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.960652+0000", 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1590999999999998e-05 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 57/31 ECSubWrite(tid=66, reqid=osd.7.0:66, at_version=57'44, trim_to=0'0, roll_forward_to=56'42))", 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:22.636576+0000", 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.3737522130000004, 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0044709479999999998, 2024-04-18T23:11:27.074 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.636576+0000", 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.636576+0000", 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.075 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.636576+0000", 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3000000000000001e-07 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.636644+0000", 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8114999999999994e-05 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.636674+0000", 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0306000000000001e-05 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.636697+0000", 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2103000000000002e-05 2024-04-18T23:11:27.076 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.636876+0000", 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017938900000000001 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.636907+0000", 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0889000000000002e-05 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.640963+0000", 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0040562319999999999 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.077 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.641046+0000", 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3384e-05 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 57/31 ECSubWrite(tid=67, reqid=osd.7.0:66, at_version=0'0, trim_to=0'0, roll_forward_to=57'44))", 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:22.839825+0000", 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.1705025229999997, 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101693535, 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.078 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.839825+0000", 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.839825+0000", 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.079 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.839827+0000", 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6360000000000001e-06 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.839851+0000", 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3983000000000001e-05 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.839891+0000", 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0435000000000002e-05 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.839916+0000", 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5210000000000001e-05 2024-04-18T23:11:27.080 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.940260+0000", 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10034370300000001 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.940277+0000", 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6779999999999999e-05 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.941448+0000", 2024-04-18T23:11:27.081 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00117062 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.941519+0000", 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1167999999999994e-05 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 60/31 ECSubWrite(tid=38, reqid=client.4568.0:452, at_version=0'0, trim_to=0'0, roll_forward_to=60'10))", 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.355307+0000", 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.655020716, 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101367186, 2024-04-18T23:11:27.082 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.355307+0000", 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.355307+0000", 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.083 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.355308+0000", 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2100000000000001e-06 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.355322+0000", 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4148000000000001e-05 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.355340+0000", 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7609999999999999e-05 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.455450+0000", 2024-04-18T23:11:27.084 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100110292 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.455551+0000", 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100593 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.455571+0000", 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9922e-05 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.456630+0000", 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010594319999999999 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.085 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.456674+0000", 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3979e-05 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 61/31 ECSubWrite(tid=40, reqid=osd.11.0:40, at_version=61'12, trim_to=0'0, roll_forward_to=60'10))", 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.654132+0000", 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.356196138, 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003883226, 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.086 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654132+0000", 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654132+0000", 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.087 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654133+0000", 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0669999999999999e-06 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654160+0000", 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7608000000000001e-05 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654176+0000", 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5359e-05 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654190+0000", 2024-04-18T23:11:27.088 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4796e-05 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654319+0000", 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012809 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654333+0000", 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4263e-05 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.657976+0000", 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036433720000000002 2024-04-18T23:11:27.089 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.658015+0000", 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8671000000000003e-05 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.090 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:27.091 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:11:27.102 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:27.098+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:27.157 INFO:tasks.rados.rados.0.smithi142.stdout:192: finishing copy_from to smithi14230322-46 2024-04-18T23:11:27.157 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 46 v 21 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-18T23:11:27.203 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:27.202+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:27.263 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:27.263 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:27.263 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:11:27.264 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:27.278 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:11:27.304 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:27.304+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:27.405 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:27.402+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:27.445 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:27.446 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:27.446 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:27.446 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:27.446 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:27.460 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:11:27.461 INFO:tasks.rados.rados.0.smithi142.stdout:191: finishing copy_from to smithi14230322-25 2024-04-18T23:11:27.461 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 25 v 15 (ObjNum 56 snap 4 seq_num 56) dirty exists 2024-04-18T23:11:27.506 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:27.502+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:27.560 INFO:tasks.rados.rados.0.smithi142.stdout:189: done (4 left) 2024-04-18T23:11:27.560 INFO:tasks.rados.rados.0.smithi142.stdout:190: done (3 left) 2024-04-18T23:11:27.560 INFO:tasks.rados.rados.0.smithi142.stdout:191: done (2 left) 2024-04-18T23:11:27.560 INFO:tasks.rados.rados.0.smithi142.stdout:192: done (1 left) 2024-04-18T23:11:27.560 INFO:tasks.rados.rados.0.smithi142.stdout:193: done (0 left) 2024-04-18T23:11:27.560 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 32 current snap is 17 2024-04-18T23:11:27.560 INFO:tasks.rados.rados.0.smithi142.stdout:194: seq_num 82 ranges {98304=622592,720896=475136,1196032=294912} 2024-04-18T23:11:27.561 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-04-18T23:11:27.568 INFO:tasks.rados.rados.0.smithi142.stdout:194: writing smithi14230322-32 from 98304 to 720896 tid 1 2024-04-18T23:11:27.573 INFO:tasks.rados.rados.0.smithi142.stdout:194: writing smithi14230322-32 from 720896 to 1196032 tid 2 2024-04-18T23:11:27.577 INFO:tasks.rados.rados.0.smithi142.stdout:194: writing smithi14230322-32 from 1196032 to 1490944 tid 3 2024-04-18T23:11:27.577 INFO:tasks.rados.rados.0.smithi142.stdout:195: snap_create 2024-04-18T23:11:27.583 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:27.583 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:27.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:273 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.001261358000001, 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.781474428, 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 273 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581592+0000", 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999999e-06 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581611+0000", 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8736000000000002e-05 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581652+0000", 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1421999999999997e-05 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681775+0000", 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100122909 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681951+0000", 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000175358 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681971+0000", 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0795e-05 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.858775+0000", 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17680337099999999 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.858953+0000", 2024-04-18T23:11:27.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017833000000000001 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.363063+0000", 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50410986700000004 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:274 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.901015249, 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.68252750799999995, 2024-04-18T23:11:27.588 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 274 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681837+0000", 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.756e-06 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681852+0000", 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4566000000000001e-05 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681881+0000", 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8651999999999999e-05 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.781982+0000", 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100101015 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782144+0000", 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162347 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782164+0000", 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9888999999999999e-05 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.959687+0000", 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17752280100000001 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.959899+0000", 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021188899999999999 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364362+0000", 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40446359300000001 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:275 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.800830834999999, 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.78490300599999996, 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:27.593 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 275 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782021+0000", 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.497e-06 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782031+0000", 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8730000000000002e-06 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782049+0000", 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8016000000000002e-05 2024-04-18T23:11:27.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782060+0000", 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1066e-05 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782264+0000", 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020450899999999999 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782285+0000", 2024-04-18T23:11:27.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1066000000000001e-05 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364492+0000", 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58220711199999997 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364799+0000", 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030637600000000002 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.566922+0000", 2024-04-18T23:11:27.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20212349099999999 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:341 3.9s0 3:906aad0a:::smithi14230322-27:head [append 0~0] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.724969+0000", 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.857881211, 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.60935175900000005, 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.598 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 341 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724969+0000", 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724969+0000", 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724971+0000", 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4899999999999999e-06 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724985+0000", 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.418e-05 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725014+0000", 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9016000000000001e-05 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725034+0000", 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9825000000000001e-05 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725162+0000", 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012797000000000001 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725175+0000", 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2255999999999999e-05 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.601 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828411+0000", 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103236438 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828613+0000", 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020252399999999999 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830448+0000", 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00183472 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.932371+0000", 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101922922 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.334124+0000", 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40175268400000003 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.334321+0000", 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000196734 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:342 3.9s0 3:906aad0a:::smithi14230322-27:head [rollback 4] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.725072+0000", 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.857777904000001, 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71010575600000003, 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 342 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725072+0000", 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725072+0000", 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725074+0000", 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-04-18T23:11:27.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725081+0000", 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9959999999999996e-06 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725098+0000", 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7785999999999999e-05 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725112+0000", 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.403e-05 2024-04-18T23:11:27.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725303+0000", 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019035999999999999 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725342+0000", 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8769000000000001e-05 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.928317+0000", 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20297505199999999 2024-04-18T23:11:27.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.928568+0000", 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025112799999999998 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.929483+0000", 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000915371 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.932373+0000", 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002890218 2024-04-18T23:11:27.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.434969+0000", 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50259540999999996 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435178+0000", 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209156 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.609 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:343 3.9s0 3:906aad0a:::smithi14230322-27:head [append 0~0] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.725142+0000", 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.85770782, 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71044070299999995, 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 343 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725142+0000", 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725142+0000", 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725144+0000", 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.347e-06 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725169+0000", 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5683000000000001e-05 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725187+0000", 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8235000000000001e-05 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725203+0000", 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5933000000000001e-05 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725399+0000", 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019563999999999999 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725408+0000", 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0999999999999993e-06 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.930345+0000", 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.204937228 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.930628+0000", 2024-04-18T23:11:27.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000282274 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.931715+0000", 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010869009999999999 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.233078+0000", 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30136315400000002 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:27.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435450+0000", 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20237229500000001 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435583+0000", 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013291299999999999 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 47/31 ECSubWriteReply(tid=80, last_complete=47'32, committed=1, applied=1))", 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.947474+0000", 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.635376031, 2024-04-18T23:11:27.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.48772908199999998, 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947474+0000", 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947474+0000", 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947475+0000", 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1829999999999999e-06 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947482+0000", 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7870000000000004e-06 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947500+0000", 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7669e-05 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.047588+0000", 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.1000887 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.434720+0000", 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38713160200000002 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435203+0000", 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048314099999999997 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.618 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:421 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~557056] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.533981+0000", 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0488687590000003, 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81687969000000005, 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 421 2024-04-18T23:11:27.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.533981+0000", 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.533981+0000", 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.533984+0000", 2024-04-18T23:11:27.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.534844+0000", 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085963299999999995 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.534879+0000", 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5305999999999997e-05 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.534898+0000", 2024-04-18T23:11:27.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.535053+0000", 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015493200000000001 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.535070+0000", 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6682999999999999e-05 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.640162+0000", 2024-04-18T23:11:27.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105092212 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.640474+0000", 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031151200000000001 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643548+0000", 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003074181 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.652446+0000", 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0088978619999999994 2024-04-18T23:11:27.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350532+0000", 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.69808661400000005 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350861+0000", 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000328585 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:422 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~671744] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:27.624 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.537766+0000", 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0450835100000004, 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81361762999999998, 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 422 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.537766+0000", 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.537766+0000", 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.537769+0000", 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1059999999999998e-06 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538649+0000", 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088010300000000003 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538679+0000", 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0263000000000001e-05 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538694+0000", 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4936000000000001e-05 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538836+0000", 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141643 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538849+0000", 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3013000000000001e-05 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.646583+0000", 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107734542 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.646882+0000", 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029901899999999998 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.648879+0000", 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019973460000000001 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.653818+0000", 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049380880000000002 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.351099+0000", 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.69728165900000005 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.351384+0000", 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028491200000000001 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:423 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~98304] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:27.630 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.538732+0000", 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0441180159999996, 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.91325199599999995, 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 423 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538732+0000", 2024-04-18T23:11:27.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538732+0000", 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538733+0000", 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538808+0000", 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4623999999999996e-05 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538822+0000", 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.401e-05 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538832+0000", 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538898+0000", 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6571999999999999e-05 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538907+0000", 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.003e-06 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.650070+0000", 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111163027 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.650383+0000", 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000312844 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.651521+0000", 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001137817 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.654205+0000", 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026839479999999998 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.451402+0000", 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.79719673599999996 2024-04-18T23:11:27.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.451984+0000", 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00058231199999999998 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:424 3.0s0 3:063b0705:::smithi14230322-16:head [setxattr _header (57) in=7b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.538853+0000", 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0439967259999996, 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.014298884, 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.636 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 424 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538853+0000", 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538853+0000", 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538854+0000", 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7300000000000004e-07 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538859+0000", 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6299999999999997e-06 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538869+0000", 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0302999999999999e-05 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538881+0000", 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2196e-05 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538998+0000", 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116821 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:27.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.539024+0000", 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5392e-05 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.652483+0000", 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.113459882 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.652660+0000", 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000176522 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:27.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.653461+0000", 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000801299 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.655473+0000", 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020117630000000002 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.552510+0000", 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.89703736599999995 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.553152+0000", 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00064173699999999997 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:425 3.0s0 3:063b0705:::smithi14230322-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.538897+0000", 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.0439531000000004, 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.219395432, 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:27.642 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 425 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538897+0000", 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538897+0000", 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538898+0000", 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8599999999999997e-07 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538901+0000", 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6169999999999998e-06 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538912+0000", 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0412e-05 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538920+0000", 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5569999999999996e-06 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.638222+0000", 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099301983999999996 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.638243+0000", 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0825000000000001e-05 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.654027+0000", 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015784231999999999 2024-04-18T23:11:27.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.654189+0000", 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000161686 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.654544+0000", 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.000355017 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.654740+0000", 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019602899999999999 2024-04-18T23:11:27.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.758292+0000", 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10355228700000001 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=94, last_complete=56'16, committed=1, applied=1))", 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.752292+0000", 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.8305579959999996, 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497650382, 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.647 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752292+0000", 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752292+0000", 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752294+0000", 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.767e-06 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752304+0000", 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9520000000000006e-06 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752329+0000", 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852428+0000", 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100098916 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.249913+0000", 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39748424700000001 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.249942+0000", 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9757e-05 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=95, last_complete=56'17, committed=1, applied=1))", 2024-04-18T23:11:27.650 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.852480+0000", 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.7303695030000004, 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49763560899999998, 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852480+0000", 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852480+0000", 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852482+0000", 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0470000000000001e-06 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852489+0000", 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7100000000000007e-06 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852510+0000", 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1109000000000001e-05 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852521+0000", 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.11e-05 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350081+0000", 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49755995800000002 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350116+0000", 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4684999999999997e-05 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=96, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.7303009510000003, 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49767800099999998, 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000001e-07 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852553+0000", 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8999999999999999e-06 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852566+0000", 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3063e-05 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852574+0000", 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5440000000000001e-06 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350198+0000", 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497624179 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350227+0000", 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8901999999999999e-05 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=97, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.853074+0000", 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.7297762490000004, 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49724826700000002, 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.657 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853074+0000", 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853074+0000", 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853075+0000", 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999999999999999e-06 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853082+0000", 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1999999999999997e-06 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853100+0000", 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357000000000001e-05 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853111+0000", 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0389e-05 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350293+0000", 2024-04-18T23:11:27.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49718215799999999 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350322+0000", 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8963e-05 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=94, last_complete=56'16, committed=1, applied=1))", 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.916991+0000", 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6658585639999997, 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.433895804, 2024-04-18T23:11:27.660 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916991+0000", 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916991+0000", 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916993+0000", 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9069999999999999e-06 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917006+0000", 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2679e-05 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917041+0000", 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5435999999999998e-05 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917063+0000", 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1080000000000001e-05 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350388+0000", 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.43332507799999997 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350887+0000", 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00049962400000000005 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.663 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=95, last_complete=56'17, committed=1, applied=1))", 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.917098+0000", 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6657516299999999, 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.434303033, 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917098+0000", 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917098+0000", 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917099+0000", 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.54e-07 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917104+0000", 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6659999999999999e-06 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917119+0000", 2024-04-18T23:11:27.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5273000000000001e-05 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917128+0000", 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5229999999999995e-06 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350964+0000", 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.43383687500000001 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.351401+0000", 2024-04-18T23:11:27.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00043694200000000001 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=96, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.917214+0000", 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6656359030000001, 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53479636500000005, 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917214+0000", 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917214+0000", 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917215+0000", 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917220+0000", 2024-04-18T23:11:27.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7299999999999996e-06 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917235+0000", 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5273000000000001e-05 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917244+0000", 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0359999999999995e-06 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.451137+0000", 2024-04-18T23:11:27.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53389361899999999 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.452010+0000", 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087298699999999998 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=97, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.917619+0000", 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.6652307640000004, 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.636734154, 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:27.670 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917619+0000", 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917619+0000", 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917620+0000", 2024-04-18T23:11:27.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9399999999999993e-07 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917627+0000", 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8360000000000003e-06 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917645+0000", 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7566000000000001e-05 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.017700+0000", 2024-04-18T23:11:27.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100055057 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.552241+0000", 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53454112899999995 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.554353+0000", 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021125720000000001 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:27.673 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:27.675 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:11:27.676 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:27.675+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:27.776 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:27.772+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:27.807 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:27.807 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:27.807 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:27.807 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:27.822 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:11:27.877 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:27.875+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:27.884 INFO:tasks.rados.rados.0.smithi142.stdout:194: finishing write tid 1 to smithi14230322-32 2024-04-18T23:11:27.977 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:27.974+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:27.984 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:27.984 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:27.984 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:27.984 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:27.984 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:27.998 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:11:28.078 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:28.078+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:28.084 INFO:tasks.rados.rados.0.smithi142.stdout:194: finishing write tid 2 to smithi14230322-32 2024-04-18T23:11:28.084 INFO:tasks.rados.rados.0.smithi142.stdout:194: finishing write tid 3 to smithi14230322-32 2024-04-18T23:11:28.084 INFO:tasks.rados.rados.0.smithi142.stdout:194: finishing write tid 4 to smithi14230322-32 2024-04-18T23:11:28.168 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:28.168 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s3 34/31 ECSubWrite(tid=0, reqid=client.4568.0:7, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.527180+0000", 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 31.641385553999999, 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0053240450000000003, 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.169 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527180+0000", 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527180+0000", 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527181+0000", 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5400000000000002e-07 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527299+0000", 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000118287 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527328+0000", 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9254000000000001e-05 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527351+0000", 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3119e-05 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527480+0000", 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012815299999999999 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.527498+0000", 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8261e-05 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532392+0000", 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0048938000000000002 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.172 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.532504+0000", 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112817 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 34/31 ECSubWrite(tid=1, reqid=client.4568.0:29, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:10:56.624154+0000", 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 31.544411475, 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0099604889999999995, 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.173 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624154+0000", 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624154+0000", 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624155+0000", 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6700000000000003e-07 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.174 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624183+0000", 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7778999999999999e-05 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624208+0000", 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5040000000000001e-05 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.624245+0000", 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7076999999999998e-05 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.632990+0000", 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0087455689999999999 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.633000+0000", 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8810000000000008e-06 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634084+0000", 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010833990000000001 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:10:56.634115+0000", 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1177000000000003e-05 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(osd.5.21:2 3.as0 3:53611365:::smithi14230322-34:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.645176+0000", 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.523389901000002, 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20476035200000001, 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645176+0000", 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645176+0000", 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645179+0000", 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.92e-06 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.178 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645190+0000", 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0947e-05 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645212+0000", 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2075000000000001e-05 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645227+0000", 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4989e-05 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645346+0000", 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011948800000000001 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645498+0000", 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015173899999999999 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.849936+0000", 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20443819399999999 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.180 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/31 ECSubReadReply(tid=8, attrs_read=0))", 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647408+0000", 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.521157940999998, 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10186719900000001, 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647408+0000", 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647408+0000", 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647409+0000", 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2440000000000001e-06 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648186+0000", 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00077665600000000003 2024-04-18T23:11:28.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648203+0000", 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6656000000000001e-05 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648218+0000", 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5085999999999999e-05 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749203+0000", 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100984977 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749275+0000", 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.258e-05 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/31 ECSubReadReply(tid=8, attrs_read=0))", 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.648647+0000", 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.519918780000001, 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100859843, 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.184 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648647+0000", 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648647+0000", 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648648+0000", 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.185 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648917+0000", 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026861399999999997 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648939+0000", 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1769000000000001e-05 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648956+0000", 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7476e-05 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.186 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749394+0000", 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100437887 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749507+0000", 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000112717 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/31 ECSubReadReply(tid=8, attrs_read=0))", 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.519703044, 2024-04-18T23:11:28.187 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20110777299999999, 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.188 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648921+0000", 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8091000000000003e-05 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.189 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648981+0000", 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0117000000000001e-05 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.849495+0000", 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.200514265 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.849971+0000", 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00047530000000000001 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.190 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.9s3 36/31 ECSubRead(tid=37, to_read={3:906aad0a:::smithi14230322-27:head=0,110592,0}, subchunks={3:906aad0a:::smithi14230322-27:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.649071+0000", 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.519494747, 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101279314, 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649071+0000", 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.191 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649071+0000", 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649072+0000", 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0699999999999996e-07 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649089+0000", 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7071999999999998e-05 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.649128+0000", 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9168999999999997e-05 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749239+0000", 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100111065 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749398+0000", 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015834000000000001 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.750350+0000", 2024-04-18T23:11:28.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000952961 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 36/31 ECSubWrite(tid=27, reqid=client.4568.0:187, at_version=36'15, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.751178+0000", 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.417387952999999, 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10144216, 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.194 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751178+0000", 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751178+0000", 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751179+0000", 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.333e-06 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751215+0000", 2024-04-18T23:11:28.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5611999999999997e-05 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751244+0000", 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9422000000000001e-05 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751267+0000", 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2779e-05 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.851479+0000", 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10021179400000001 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.196 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.851497+0000", 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8128999999999999e-05 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.852539+0000", 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010423380000000001 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.852620+0000", 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0753000000000004e-05 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.197 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s3 42/31 ECSubWrite(tid=63, reqid=osd.14.0:61, at_version=0'0, trim_to=0'0, roll_forward_to=42'25))", 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:07.263035+0000", 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 20.905531394, 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10154968, 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263035+0000", 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.198 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263035+0000", 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263037+0000", 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1069999999999999e-06 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263061+0000", 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4131999999999999e-05 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.199 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263092+0000", 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1399000000000002e-05 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.263109+0000", 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.694e-05 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.363418+0000", 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100308642 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.363434+0000", 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6124999999999999e-05 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.364491+0000", 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010569080000000001 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:07.364584+0000", 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3426999999999998e-05 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.201 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 46/31 ECSubWrite(tid=41, reqid=client.4568.0:323, at_version=0'0, trim_to=0'0, roll_forward_to=46'23))", 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:11.424855+0000", 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.743710812, 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.1019087, 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424855+0000", 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.202 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424855+0000", 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424857+0000", 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2469999999999998e-06 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424910+0000", 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2784000000000001e-05 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424957+0000", 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7141999999999997e-05 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.425023+0000", 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5814000000000004e-05 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525324+0000", 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100300527 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525339+0000", 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5523999999999999e-05 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.526710+0000", 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001370511 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.526764+0000", 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4150999999999997e-05 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.205 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 46/31 ECSubWrite(tid=42, reqid=client.4568.0:330, at_version=46'25, trim_to=0'0, roll_forward_to=46'23))", 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:11.431071+0000", 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.737494471000002, 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.096741165000000004, 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431071+0000", 2024-04-18T23:11:28.206 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431071+0000", 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431073+0000", 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5230000000000001e-06 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431138+0000", 2024-04-18T23:11:28.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4714000000000004e-05 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431167+0000", 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9456000000000001e-05 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.431190+0000", 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3013000000000001e-05 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525828+0000", 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.094638112999999996 2024-04-18T23:11:28.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525842+0000", 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3963e-05 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.527743+0000", 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001900765 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.527813+0000", 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9617999999999997e-05 2024-04-18T23:11:28.209 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.ds1 47/31 ECSubRead(tid=46, to_read={3:ba4438ef:::smithi14230322-48:head=0,8192,0}, subchunks={3:ba4438ef:::smithi14230322-48:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:12.743722+0000", 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.424843542, 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100400477, 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.210 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.743722+0000", 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.743722+0000", 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.743723+0000", 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8599999999999998e-07 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.743731+0000", 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3999999999999992e-06 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.743749+0000", 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7163000000000002e-05 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.743759+0000", 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0094e-05 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.843923+0000", 2024-04-18T23:11:28.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10016388800000001 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.844123+0000", 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020024600000000001 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s3 47/31 ECSubWrite(tid=79, reqid=client.4568.0:341, at_version=47'31, trim_to=0'0, roll_forward_to=45'29))", 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:12.829509+0000", 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.339057241000001, 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10224200899999999, 2024-04-18T23:11:28.213 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829509+0000", 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829509+0000", 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.214 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829509+0000", 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3300000000000003e-07 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829546+0000", 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6777999999999999e-05 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829574+0000", 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7645999999999999e-05 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.215 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.929675+0000", 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100101352 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.929815+0000", 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000140161 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.929836+0000", 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1015000000000001e-05 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.216 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.931716+0000", 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018800469999999999 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.931751+0000", 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4576999999999998e-05 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s2 47/31 ECSubWrite(tid=191, reqid=client.4568.0:345, at_version=47'22, trim_to=0'0, roll_forward_to=45'20))", 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:13.038481+0000", 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.130085226, 2024-04-18T23:11:28.217 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0078103080000000002, 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.038481+0000", 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.038481+0000", 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.218 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.038482+0000", 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.167e-06 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.038505+0000", 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2793999999999999e-05 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.038520+0000", 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5528000000000001e-05 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.219 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.038529+0000", 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6950000000000006e-06 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.038618+0000", 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8760999999999999e-05 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.038635+0000", 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7844999999999999e-05 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.220 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.046206+0000", 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0075707270000000002 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.046291+0000", 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4790999999999995e-05 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 51/31 ECSubWrite(tid=50, reqid=client.4568.0:389, at_version=51'31, trim_to=0'0, roll_forward_to=47'29))", 2024-04-18T23:11:28.221 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.876486+0000", 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.292080138999999, 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10451489899999999, 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876486+0000", 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876486+0000", 2024-04-18T23:11:28.222 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876486+0000", 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9999999999999997e-07 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876784+0000", 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029813000000000002 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876821+0000", 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6702999999999997e-05 2024-04-18T23:11:28.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.976922+0000", 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100100598 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.977083+0000", 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016119800000000001 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.977113+0000", 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0273e-05 2024-04-18T23:11:28.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980943+0000", 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038295320000000001 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.981001+0000", 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7865000000000002e-05 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.225 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.ds1 56/31 ECSubRead(tid=61, to_read={3:ba4438ef:::smithi14230322-48:head=0,8192,0}, subchunks={3:ba4438ef:::smithi14230322-48:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:21.743138+0000", 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.4254281669999997, 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100643019, 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.743138+0000", 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.743138+0000", 2024-04-18T23:11:28.226 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.743138+0000", 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9299999999999999e-07 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.743149+0000", 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1279e-05 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.743186+0000", 2024-04-18T23:11:28.227 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7072e-05 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.743206+0000", 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9344e-05 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.843490+0000", 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10028405 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.843781+0000", 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000290881 2024-04-18T23:11:28.228 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.ds1 56/31 ECSubRead(tid=62, to_read={3:ba4438ef:::smithi14230322-48:head=0,8192,0}, subchunks={3:ba4438ef:::smithi14230322-48:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:21.744963+0000", 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.4236029099999996, 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099183071999999997, 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.744963+0000", 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.229 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.744963+0000", 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.744965+0000", 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5400000000000001e-06 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.744980+0000", 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5345999999999998e-05 2024-04-18T23:11:28.230 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.745015+0000", 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5012999999999998e-05 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.745033+0000", 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8029e-05 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.843924+0000", 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098891282999999996 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.231 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.844146+0000", 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022186100000000001 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.ds1 56/31 ECSubRead(tid=63, to_read={3:ba4438ef:::smithi14230322-48:head=0,8192,0}, subchunks={3:ba4438ef:::smithi14230322-48:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:21.746153+0000", 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.4224126879999996, 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098302010999999995, 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.232 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.746153+0000", 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.746153+0000", 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.746155+0000", 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.657e-06 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.233 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.746171+0000", 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5786000000000001e-05 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.746200+0000", 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9295999999999999e-05 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.746224+0000", 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3629e-05 2024-04-18T23:11:28.234 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.844267+0000", 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098043715000000004 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.844455+0000", 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018792799999999999 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.235 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 56/31 ECSubWrite(tid=64, reqid=client.4568.0:426, at_version=56'42, trim_to=0'0, roll_forward_to=54'40))", 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:21.953759+0000", 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.2148065529999998, 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0061634300000000001, 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.953759+0000", 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.236 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.953759+0000", 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.953761+0000", 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1270000000000001e-06 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.954286+0000", 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00052464799999999995 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.237 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.954334+0000", 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8307999999999999e-05 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.954368+0000", 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3099999999999998e-05 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.954554+0000", 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018598399999999999 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.238 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.954633+0000", 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9480999999999993e-05 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.959829+0000", 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.005196155 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.959923+0000", 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3627000000000003e-05 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.239 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 61/31 ECSubWrite(tid=70, reqid=osd.7.0:70, at_version=61'48, trim_to=0'0, roll_forward_to=58'46))", 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:26.750469+0000", 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 1.418097156, 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10194142, 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750469+0000", 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.240 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750469+0000", 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750469+0000", 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8299999999999998e-07 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750492+0000", 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3247999999999998e-05 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.241 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750511+0000", 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8790000000000001e-05 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.850592+0000", 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100080628 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.850722+0000", 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013027199999999999 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.242 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.850739+0000", 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6515999999999999e-05 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.852244+0000", 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015051000000000001 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.852410+0000", 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016648300000000001 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.243 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.244 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:28.244 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:28.244 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:28.245 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:11:28.246 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:28.242+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:28.288 INFO:tasks.rados.rados.0.smithi142.stdout:194: finishing write tid 5 to smithi14230322-32 2024-04-18T23:11:28.288 INFO:tasks.rados.rados.0.smithi142.stdout:194: oid 32 updating version 0 to 92 2024-04-18T23:11:28.288 INFO:tasks.rados.rados.0.smithi142.stdout:194: oid 32 version 92 is already newer than 90 2024-04-18T23:11:28.288 INFO:tasks.rados.rados.0.smithi142.stdout:194: oid 32 version 92 is already newer than 91 2024-04-18T23:11:28.288 INFO:tasks.rados.rados.0.smithi142.stdout:194: oid 32 version 92 is already newer than 89 2024-04-18T23:11:28.288 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 32 v 92 (ObjNum 82 snap 17 seq_num 82) dirty exists 2024-04-18T23:11:28.288 INFO:tasks.rados.rados.0.smithi142.stdout:194: left oid 32 (ObjNum 82 snap 17 seq_num 82) 2024-04-18T23:11:28.345 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:28.342+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:28.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:28.412 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:28.413 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:28.413 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:28.413 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:28.429 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:11:28.446 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:28.442+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:28.502 INFO:tasks.rados.rados.0.smithi142.stdout:194: done (1 left) 2024-04-18T23:11:28.502 INFO:tasks.rados.rados.0.smithi142.stdout:195: done (0 left) 2024-04-18T23:11:28.502 INFO:tasks.rados.rados.0.smithi142.stdout:196: snap_create 2024-04-18T23:11:28.529 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:28.529 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:28.547 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:28.542+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:28.547 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:28.548 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:28.548 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:28.548 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:28.548 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:28.550 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:28.550 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:28.562 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:11:28.573 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:28.573 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:28.592 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:28.592 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:28.611 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:28.611 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:28.631 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:28.632 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:28.648 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:28.647+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:28.652 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:28.652 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:28.674 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:28.674 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:28.684 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:28.684 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:28.693 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:28.693 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:28.699 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:11:28.714 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:28.714 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:28.735 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:28.735 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:28.748 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:28.746+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:28.757 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:28.773 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:28.774 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:28.794 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:28.795 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:28.816 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:28.817 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENXIO: problem getting command descriptions from osd.15 2024-04-18T23:11:28.817 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:28.818 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:28.821 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T23:11:28.821 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T23:11:28.832 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:11:28.878 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:28.876+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:28.941 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:28.942 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:28.950 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:28.946+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:28.954 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:11:29.051 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:29.051+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:29.115 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:29.116 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:29.130 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:11:29.152 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:29.151+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:29.253 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:29.250+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:29.261 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:29.261 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:29.275 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:11:29.354 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:29.350+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:29.393 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:29.393 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:29.408 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:11:29.455 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:29.456+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:29.503 INFO:tasks.rados.rados.0.smithi142.stdout:196: done (0 left) 2024-04-18T23:11:29.503 INFO:tasks.rados.rados.0.smithi142.stdout:197: snap_remove snap 14 2024-04-18T23:11:29.530 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:29.530 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:29.544 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:11:29.556 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:29.552+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:29.658 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:29.655+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:29.668 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:29.668 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:29.675 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T23:11:29.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:11:29.772 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:29.770+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:29.843 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:29.843 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:29.858 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:11:29.900 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:29.898+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:29.961 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:29.958+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:30.015 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:30.015 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:30.028 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:11:30.062 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:30.060+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:30.163 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:30.158+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:30.192 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:30.192 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:30.206 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:11:30.264 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:30.263+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:30.366 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:30.366 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:30.366 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:30.367+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:30.379 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:11:30.467 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:30.466+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:30.495 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:30.495 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:30.508 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-04-18T23:11:30.509 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:11:30.510 INFO:tasks.rados.rados.0.smithi142.stdout:197: done (0 left) 2024-04-18T23:11:30.510 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 22 current snap is 19 2024-04-18T23:11:30.510 INFO:tasks.rados.rados.0.smithi142.stdout:198: seq_num 83 ranges {360448=638976,999424=622592,1622016=147456} 2024-04-18T23:11:30.525 INFO:tasks.rados.rados.0.smithi142.stdout:198: writing smithi14230322-22 from 360448 to 999424 tid 1 2024-04-18T23:11:30.536 INFO:tasks.rados.rados.0.smithi142.stdout:198: writing smithi14230322-22 from 999424 to 1622016 tid 2 2024-04-18T23:11:30.538 INFO:tasks.rados.rados.0.smithi142.stdout:198: writing smithi14230322-22 from 1622016 to 1769472 tid 3 2024-04-18T23:11:30.538 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 10 current snap is 19 2024-04-18T23:11:30.538 INFO:tasks.rados.rados.0.smithi142.stdout:199: seq_num 84 ranges {638976=655360,1294336=638976,1933312=131072} 2024-04-18T23:11:30.558 INFO:tasks.rados.rados.0.smithi142.stdout:199: writing smithi14230322-10 from 638976 to 1294336 tid 1 2024-04-18T23:11:30.568 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:30.566+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:30.568 INFO:tasks.rados.rados.0.smithi142.stdout:199: writing smithi14230322-10 from 1294336 to 1933312 tid 2 2024-04-18T23:11:30.570 INFO:tasks.rados.rados.0.smithi142.stdout:199: writing smithi14230322-10 from 1933312 to 2064384 tid 3 2024-04-18T23:11:30.570 INFO:tasks.rados.rados.0.smithi142.stdout:200: setattr oid 6 current snap is 19 2024-04-18T23:11:30.574 INFO:tasks.rados.rados.0.smithi142.stdout:201: read oid 32 snap 15 2024-04-18T23:11:30.574 INFO:tasks.rados.rados.0.smithi142.stdout:201: expect (ObjNum 5 snap 0 seq_num 5) 2024-04-18T23:11:30.574 INFO:tasks.rados.rados.0.smithi142.stdout:202: snap_create 2024-04-18T23:11:30.631 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:30.632 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:30.646 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:11:30.669 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:30.667+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:30.714 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:30.712+0000 7f742b85f7c0 -1 osd.15 48 log_to_monitors true 2024-04-18T23:11:30.769 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:30.770 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:30.770 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:30.766+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:30.778 INFO:tasks.rados.rados.0.smithi142.stdout:199: finishing write tid 1 to smithi14230322-10 2024-04-18T23:11:30.786 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:11:30.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:30.872 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:30.870+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:30.877 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 6 v 96 (ObjNum 5 snap 0 seq_num 5) dirty exists 2024-04-18T23:11:30.879 INFO:tasks.rados.rados.0.smithi142.stdout:199: finishing write tid 2 to smithi14230322-10 2024-04-18T23:11:30.946 INFO:tasks.rados.rados.0.smithi142.stdout:198: finishing write tid 1 to smithi14230322-22 2024-04-18T23:11:30.947 INFO:tasks.rados.rados.0.smithi142.stdout:198: finishing write tid 2 to smithi14230322-22 2024-04-18T23:11:30.948 INFO:tasks.rados.rados.0.smithi142.stdout:198: finishing write tid 3 to smithi14230322-22 2024-04-18T23:11:30.949 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:30.949 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:30.962 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:11:30.964 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:30.964 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:30.972 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:30.970+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:30.980 INFO:tasks.rados.rados.0.smithi142.stdout:199: finishing write tid 3 to smithi14230322-10 2024-04-18T23:11:30.980 INFO:tasks.rados.rados.0.smithi142.stdout:199: finishing write tid 4 to smithi14230322-10 2024-04-18T23:11:30.984 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:30.984 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:31.005 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:31.005 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:31.026 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:31.026 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:31.072 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:31.070+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:31.093 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:31.093 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:31.093 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:31.093 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:31.098 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:31.098 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:31.107 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:11:31.125 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:31.125 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:31.154 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:31.154 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:31.173 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:31.168+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:31.178 INFO:tasks.rados.rados.0.smithi142.stdout:201: expect (ObjNum 5 snap 0 seq_num 5) 2024-04-18T23:11:31.184 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:31.184 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:31.194 INFO:tasks.rados.rados.0.smithi142.stdout:199: finishing write tid 5 to smithi14230322-10 2024-04-18T23:11:31.195 INFO:tasks.rados.rados.0.smithi142.stdout:199: oid 10 updating version 0 to 54 2024-04-18T23:11:31.195 INFO:tasks.rados.rados.0.smithi142.stdout:199: oid 10 updating version 54 to 57 2024-04-18T23:11:31.195 INFO:tasks.rados.rados.0.smithi142.stdout:199: oid 10 version 57 is already newer than 56 2024-04-18T23:11:31.195 INFO:tasks.rados.rados.0.smithi142.stdout:199: oid 10 version 57 is already newer than 55 2024-04-18T23:11:31.195 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 10 v 57 (ObjNum 84 snap 19 seq_num 84) dirty exists 2024-04-18T23:11:31.195 INFO:tasks.rados.rados.0.smithi142.stdout:199: left oid 10 (ObjNum 84 snap 19 seq_num 84) 2024-04-18T23:11:31.209 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:31.209 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:31.228 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:31.228 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:31.228 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:31.228 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:31.228 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:31.232 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:31.232 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:31.240 INFO:tasks.rados.rados.0.smithi142.stdout:198: finishing write tid 4 to smithi14230322-22 2024-04-18T23:11:31.243 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:11:31.258 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:31.258 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:31.284 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:31.284 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:31.286 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:31.287+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:31.306 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:31.307 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:31.327 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:31.328 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:31.368 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:31.368 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:31.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:31.368 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:31.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.368 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/31 ECSubWrite(tid=2, reqid=client.4568.0:19, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:31.368 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:31.368 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.839494162000001, 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19915530100000001, 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528004+0000", 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528243+0000", 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239244 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528271+0000", 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7535e-05 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528293+0000", 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2395000000000002e-05 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.626035+0000", 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.097742512000000004 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.626059+0000", 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3739000000000001e-05 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727090+0000", 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101030651 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727158+0000", 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8415000000000001e-05 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.372 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.829609597000001, 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103172215, 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537889+0000", 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839999999999999e-06 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537921+0000", 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2360999999999997e-05 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537948+0000", 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6421e-05 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537963+0000", 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5218999999999999e-05 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638148+0000", 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100185015 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638163+0000", 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4793e-05 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.640970+0000", 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028075439999999999 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.641060+0000", 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9377999999999997e-05 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.2s5 33/31 ECSubWrite(tid=160, reqid=client.4568.0:26, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.623173+0000", 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.744324266, 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101822397, 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623173+0000", 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623173+0000", 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623174+0000", 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.16e-07 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623206+0000", 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2660999999999998e-05 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.623229+0000", 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2481999999999999e-05 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723331+0000", 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010213699999999 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723530+0000", 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019928599999999999 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.723553+0000", 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2442e-05 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724939+0000", 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013867300000000001 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.724995+0000", 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.5742999999999998e-05 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.380 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:5, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.739408386999997, 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103548802, 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628091+0000", 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1129999999999999e-06 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628179+0000", 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7818000000000005e-05 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628219+0000", 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0373000000000001e-05 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628243+0000", 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3549000000000001e-05 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628766+0000", 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052308499999999996 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628791+0000", 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4984999999999999e-05 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731546+0000", 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102755535 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731638+0000", 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1343999999999998e-05 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=2, reqid=client.4568.0:22, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.739196231000001, 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103381863, 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628302+0000", 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628476+0000", 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017366199999999999 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628499+0000", 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3025000000000001e-05 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628519+0000", 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0194999999999999e-05 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629371+0000", 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085246300000000003 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.387 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629394+0000", 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3311000000000001e-05 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731650+0000", 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10225558899999999 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731683+0000", 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2654999999999998e-05 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.388 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=3, reqid=client.4568.0:23, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.738499040999997, 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103947704, 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629005+0000", 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7089999999999996e-06 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629040+0000", 2024-04-18T23:11:31.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5133999999999999e-05 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629064+0000", 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4422000000000001e-05 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629079+0000", 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4949e-05 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731181+0000", 2024-04-18T23:11:31.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102101339 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731199+0000", 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8175000000000001e-05 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.732877+0000", 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016778050000000001 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.732946+0000", 2024-04-18T23:11:31.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9171000000000006e-05 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 40/31 ECSubWrite(tid=19, reqid=osd.5.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:05.189920+0000", 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.177577095, 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101951974, 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189920+0000", 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189920+0000", 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189923+0000", 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6800000000000002e-06 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189945+0000", 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2002e-05 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.189977+0000", 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2122000000000002e-05 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.290109+0000", 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100132397 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.290251+0000", 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000141395 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.290277+0000", 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5959e-05 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.291785+0000", 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001507993 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:05.291872+0000", 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7425999999999999e-05 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 42/31 ECSubWrite(tid=27, reqid=osd.8.0:27, at_version=42'21, trim_to=0'0, roll_forward_to=40'19))", 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:06.598717+0000", 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.768780591999999, 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101705305, 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.397 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598717+0000", 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598717+0000", 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598717+0000", 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4299999999999998e-07 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598760+0000", 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3538000000000002e-05 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598784+0000", 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3618e-05 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.598800+0000", 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5489999999999999e-05 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.699022+0000", 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100222062 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.699037+0000", 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5290000000000001e-05 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.700359+0000", 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001322043 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.700422+0000", 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2821e-05 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 42/31 ECSubWrite(tid=87, reqid=osd.4.0:85, at_version=0'0, trim_to=0'0, roll_forward_to=42'27))", 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:06.785569+0000", 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.581928173000001, 2024-04-18T23:11:31.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101991601, 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785569+0000", 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785569+0000", 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785571+0000", 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.863e-06 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785600+0000", 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8812000000000001e-05 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.785630+0000", 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0015e-05 2024-04-18T23:11:31.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.885735+0000", 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010559199999999 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.885865+0000", 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000130087 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.885914+0000", 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8853999999999997e-05 2024-04-18T23:11:31.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.887429+0000", 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001515048 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.887561+0000", 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013133 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.405 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 42/31 ECSubWrite(tid=28, reqid=osd.8.0:27, at_version=0'0, trim_to=0'0, roll_forward_to=42'21))", 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:06.888631+0000", 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.478866479000001, 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10187332, 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888631+0000", 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888631+0000", 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888631+0000", 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4300000000000002e-07 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888653+0000", 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1259000000000001e-05 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.888684+0000", 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0885e-05 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.988782+0000", 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10009855600000001 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.989001+0000", 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021871600000000001 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.989034+0000", 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3071000000000002e-05 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.990449+0000", 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001415166 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:06.990504+0000", 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4824000000000002e-05 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 43/31 ECSubWrite(tid=30, reqid=osd.8.0:30, at_version=43'23, trim_to=0'0, roll_forward_to=42'21))", 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:07.609293+0000", 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.758203948999999, 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10199349200000001, 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609293+0000", 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609293+0000", 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609294+0000", 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4000000000000002e-07 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609357+0000", 2024-04-18T23:11:31.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3386999999999995e-05 2024-04-18T23:11:31.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609390+0000", 2024-04-18T23:11:31.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2838e-05 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.609416+0000", 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5799000000000002e-05 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.709727+0000", 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100311764 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.709776+0000", 2024-04-18T23:11:31.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8198000000000002e-05 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.711192+0000", 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00141602 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.711287+0000", 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4945999999999995e-05 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.415 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 47/31 ECSubWrite(tid=22, reqid=osd.5.0:22, at_version=47'16, trim_to=0'0, roll_forward_to=46'14))", 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.623975+0000", 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.743522124999998, 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102688324, 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.623975+0000", 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.623975+0000", 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.623976+0000", 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9299999999999997e-07 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.624030+0000", 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4165000000000001e-05 2024-04-18T23:11:31.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.624064+0000", 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4610999999999998e-05 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.724208+0000", 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10014355799999999 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.724385+0000", 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000177123 2024-04-18T23:11:31.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.724414+0000", 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8600999999999999e-05 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.726569+0000", 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002155751 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.726663+0000", 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3821999999999997e-05 2024-04-18T23:11:31.419 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 47/31 ECSubWrite(tid=116, reqid=client.4568.0:335, at_version=47'35, trim_to=0'0, roll_forward_to=47'33))", 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.935986+0000", 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.431510638999999, 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102410364, 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.935986+0000", 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.935986+0000", 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.935988+0000", 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8729999999999999e-06 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936143+0000", 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015484299999999999 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936174+0000", 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0571999999999998e-05 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036325+0000", 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10015107400000001 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036499+0000", 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017410200000000001 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036514+0000", 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4586000000000001e-05 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038304+0000", 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017909309999999999 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038397+0000", 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2383e-05 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 61/31 ECSubWrite(tid=180, reqid=osd.4.0:180, at_version=61'64, trim_to=0'0, roll_forward_to=58'62))", 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.646084+0000", 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.7214126279999995, 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102581479, 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646084+0000", 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646084+0000", 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646085+0000", 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9299999999999996e-07 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646133+0000", 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7995999999999999e-05 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646158+0000", 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4264999999999999e-05 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646180+0000", 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2523000000000001e-05 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646347+0000", 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000166768 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646375+0000", 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8056e-05 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.748576+0000", 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102201088 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.748666+0000", 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9889999999999995e-05 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 61/31 ECSubWrite(tid=183, reqid=osd.4.0:183, at_version=61'66, trim_to=0'0, roll_forward_to=58'62))", 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.647533+0000", 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.7199638960000003, 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102163163, 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647533+0000", 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647533+0000", 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647535+0000", 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.66e-06 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647589+0000", 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4106e-05 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647629+0000", 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0470999999999999e-05 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647653+0000", 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3213e-05 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.747920+0000", 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100266916 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.747935+0000", 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5656000000000001e-05 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.749603+0000", 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016682229999999999 2024-04-18T23:11:31.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.749696+0000", 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2917999999999994e-05 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 61/31 ECSubWrite(tid=42, reqid=osd.2.0:42, at_version=61'29, trim_to=0'0, roll_forward_to=60'27))", 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.658283+0000", 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.7092142739999998, 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.18780378, 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.432 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658283+0000", 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658283+0000", 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658285+0000", 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7060000000000001e-06 2024-04-18T23:11:31.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658326+0000", 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1149000000000002e-05 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658355+0000", 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9009e-05 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658371+0000", 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.59e-05 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.757618+0000", 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.099247506999999999 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.757642+0000", 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3929000000000001e-05 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.846008+0000", 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.088365851999999995 2024-04-18T23:11:31.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.846087+0000", 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8727999999999996e-05 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 61/31 ECSubWrite(tid=43, reqid=osd.2.0:43, at_version=61'31, trim_to=0'0, roll_forward_to=60'27))", 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.659216+0000", 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.7082806719999999, 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.18816092300000001, 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.436 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659216+0000", 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659216+0000", 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.437 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659218+0000", 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6559999999999999e-06 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659261+0000", 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2432000000000003e-05 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659290+0000", 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9689e-05 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.438 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659306+0000", 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5480000000000001e-05 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.845119+0000", 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.18581310400000001 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.845150+0000", 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1334999999999997e-05 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.439 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.847302+0000", 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002151802 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.847377+0000", 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5425000000000004e-05 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 61/31 ECSubWrite(tid=186, reqid=osd.4.0:186, at_version=61'68, trim_to=0'0, roll_forward_to=61'66))", 2024-04-18T23:11:31.440 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4108379900000001, 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201499973, 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:11:31.441 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956661+0000", 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7370000000000001e-06 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956723+0000", 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2564999999999995e-05 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956762+0000", 2024-04-18T23:11:31.442 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8711999999999998e-05 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.056877+0000", 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10011492900000001 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.057020+0000", 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142662 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.057046+0000", 2024-04-18T23:11:31.443 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6486e-05 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.158121+0000", 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10107499 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.158159+0000", 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7892000000000002e-05 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.444 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 61/31 ECSubWrite(tid=44, reqid=osd.2.0:43, at_version=0'0, trim_to=0'0, roll_forward_to=61'31))", 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.245329+0000", 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.1221682680000002, 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10201531899999999, 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245329+0000", 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.445 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245329+0000", 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245331+0000", 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.717e-06 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245355+0000", 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4099000000000001e-05 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.446 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245386+0000", 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1099999999999997e-05 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345534+0000", 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100148396 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345807+0000", 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027293199999999998 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.447 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345823+0000", 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5512999999999999e-05 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.347248+0000", 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001425054 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.347344+0000", 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6508000000000005e-05 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.448 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 61/31 ECSubWrite(tid=45, reqid=osd.2.0:45, at_version=61'33, trim_to=0'0, roll_forward_to=61'31))", 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.0219101779999997, 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20243294100000001, 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.449 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345589+0000", 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7540000000000001e-06 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.450 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345643+0000", 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4707999999999999e-05 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345662+0000", 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8870000000000001e-05 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345675+0000", 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2482e-05 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.545813+0000", 2024-04-18T23:11:31.451 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20013862499999999 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.545829+0000", 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5201999999999999e-05 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.547933+0000", 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021040389999999998 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.548020+0000", 2024-04-18T23:11:31.452 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7261000000000004e-05 2024-04-18T23:11:31.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.453 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.453 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:31.453 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:31.453 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:31.453 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:11:31.455 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:31.450+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:31.455 INFO:tasks.rados.rados.0.smithi142.stdout:198: finishing write tid 5 to smithi14230322-22 2024-04-18T23:11:31.455 INFO:tasks.rados.rados.0.smithi142.stdout:198: oid 22 updating version 0 to 45 2024-04-18T23:11:31.455 INFO:tasks.rados.rados.0.smithi142.stdout:198: oid 22 version 45 is already newer than 43 2024-04-18T23:11:31.455 INFO:tasks.rados.rados.0.smithi142.stdout:198: oid 22 version 45 is already newer than 44 2024-04-18T23:11:31.455 INFO:tasks.rados.rados.0.smithi142.stdout:198: oid 22 version 45 is already newer than 42 2024-04-18T23:11:31.456 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 22 v 45 (ObjNum 83 snap 19 seq_num 83) dirty exists 2024-04-18T23:11:31.456 INFO:tasks.rados.rados.0.smithi142.stdout:198: left oid 22 (ObjNum 83 snap 19 seq_num 83) 2024-04-18T23:11:31.456 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:11:31.469 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:31.469 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:31.470 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENXIO: problem getting command descriptions from osd.15 2024-04-18T23:11:31.475 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-18T23:11:31.475 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-18T23:11:31.527 INFO:tasks.rados.rados.0.smithi142.stdout:198: done (4 left) 2024-04-18T23:11:31.527 INFO:tasks.rados.rados.0.smithi142.stdout:199: done (3 left) 2024-04-18T23:11:31.527 INFO:tasks.rados.rados.0.smithi142.stdout:200: done (2 left) 2024-04-18T23:11:31.527 INFO:tasks.rados.rados.0.smithi142.stdout:201: done (1 left) 2024-04-18T23:11:31.527 INFO:tasks.rados.rados.0.smithi142.stdout:202: done (0 left) 2024-04-18T23:11:31.527 INFO:tasks.rados.rados.0.smithi142.stdout:203: rmattr oid 47 current snap is 20 2024-04-18T23:11:31.528 INFO:tasks.rados.rados.0.smithi142.stdout:203: done (0 left) 2024-04-18T23:11:31.528 INFO:tasks.rados.rados.0.smithi142.stdout:204: rmattr oid 39 current snap is 20 2024-04-18T23:11:31.528 INFO:tasks.rados.rados.0.smithi142.stdout:205: read oid 5 snap -1 2024-04-18T23:11:31.528 INFO:tasks.rados.rados.0.smithi142.stdout:205: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-18T23:11:31.528 INFO:tasks.rados.rados.0.smithi142.stdout:206: read oid 48 snap -1 2024-04-18T23:11:31.528 INFO:tasks.rados.rados.0.smithi142.stdout:206: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:31.528 INFO:tasks.rados.rados.0.smithi142.stdout:207: setattr oid 38 current snap is 20 2024-04-18T23:11:31.529 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 37 current snap is 20 2024-04-18T23:11:31.529 INFO:tasks.rados.rados.0.smithi142.stdout:208: seq_num 85 ranges {606208=671744,1277952=770048} 2024-04-18T23:11:31.538 INFO:tasks.rados.rados.0.smithi142.stdout:208: writing smithi14230322-37 from 606208 to 1277952 tid 1 2024-04-18T23:11:31.544 INFO:tasks.rados.rados.0.smithi142.stdout:208: writing smithi14230322-37 from 1277952 to 2048000 tid 2 2024-04-18T23:11:31.545 INFO:tasks.rados.rados.0.smithi142.stdout:209: copy_from oid 41 from oid 31 current snap is 20 2024-04-18T23:11:31.545 INFO:tasks.rados.rados.0.smithi142.stdout:210: snap_create 2024-04-18T23:11:31.555 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:31.554+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:no valid command found; 10 closest matches: 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:0 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:1 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:2 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:abort 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:assert 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:bluefs debug_inject_read_zeros 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:bluefs files list 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:bluefs stats 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:bluestore allocator dump block 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:bluestore allocator fragmentation block 2024-04-18T23:11:31.583 INFO:teuthology.orchestra.run.smithi142.stderr:admin_socket: invalid command 2024-04-18T23:11:31.585 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-18T23:11:31.586 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-15, ['dump_ops_in_flight'] 2024-04-18T23:11:31.626 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:31.690 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:31.690 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:31.692 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:11:31.692 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:31.691+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:31.738 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 39 v 37 (ObjNum 38 snap 0 seq_num 38) dirty exists 2024-04-18T23:11:31.792 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:31.790+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:31.840 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 38 v 98 (ObjNum 67 snap 7 seq_num 67) dirty exists 2024-04-18T23:11:31.856 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:31.856 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:31.856 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:31.856 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:31.856 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:31.870 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:11:31.914 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:31.915+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:31.944 INFO:tasks.rados.rados.0.smithi142.stdout:208: finishing write tid 1 to smithi14230322-37 2024-04-18T23:11:31.944 INFO:tasks.rados.rados.0.smithi142.stdout:208: finishing write tid 2 to smithi14230322-37 2024-04-18T23:11:31.945 INFO:tasks.rados.rados.0.smithi142.stdout:208: finishing write tid 3 to smithi14230322-37 2024-04-18T23:11:31.956 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:31.952+0000 7f7423634640 -1 osd.15 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-18T23:11:31.993 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:31.988+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:31.996 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:31.996 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 33/31 ECSubWrite(tid=1, reqid=client.4568.0:16, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.520061+0000", 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.475956148000002, 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10251751100000001, 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520061+0000", 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520061+0000", 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520063+0000", 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.95e-06 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520209+0000", 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014583700000000001 2024-04-18T23:11:31.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520243+0000", 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4334e-05 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.520266+0000", 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3118000000000001e-05 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.620586+0000", 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100320109 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:31.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.620600+0000", 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3453e-05 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622489+0000", 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018891730000000001 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.622578+0000", 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9536999999999993e-05 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.000 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:176 3.8s0 3:1090b2d1:::smithi14230322-15:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.456381393000001, 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.53110666500000003, 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 176 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539638+0000", 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3829999999999999e-06 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539651+0000", 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2549e-05 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539675+0000", 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4413000000000002e-05 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539689+0000", 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4161999999999999e-05 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539831+0000", 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014226500000000001 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op must wait for map", 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539865+0000", 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3185000000000002e-05 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644415+0000", 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10455017599999999 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644498+0000", 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3568999999999999e-05 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.865827+0000", 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.22132859799999999 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.868846+0000", 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030189309999999999 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070576+0000", 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20172967 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070742+0000", 2024-04-18T23:11:32.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016676399999999999 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:177 3.8s0 3:1090b2d1:::smithi14230322-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.456298760999999, 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32703084799999999, 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:32.006 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 177 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539719+0000", 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.37e-07 2024-04-18T23:11:32.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539723+0000", 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1620000000000001e-06 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539733+0000", 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4630000000000007e-06 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539739+0000", 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7880000000000001e-06 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540029+0000", 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029001100000000001 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540050+0000", 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1033e-05 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644678+0000", 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10462763999999999 2024-04-18T23:11:32.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644714+0000", 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6202999999999999e-05 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866546+0000", 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.22183165399999999 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866650+0000", 2024-04-18T23:11:32.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104465 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866749+0000", 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8791999999999999e-05 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:207 3.8s0 3:1c19a06b:::smithi14230322-28:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.434682594000002, 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40709379200000001, 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.011 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 207 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561336+0000", 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.638e-06 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561354+0000", 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7413000000000001e-05 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561372+0000", 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8003e-05 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561383+0000", 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1425e-05 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561495+0000", 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111661 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561506+0000", 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.113e-05 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644748+0000", 2024-04-18T23:11:32.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.083242308000000001 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644807+0000", 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8909999999999997e-05 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.854271+0000", 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.209463753 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.856288+0000", 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020173909999999999 2024-04-18T23:11:32.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.968263+0000", 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.111975256 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.968428+0000", 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016490400000000001 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:208 3.8s0 3:1c19a06b:::smithi14230322-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:32.016 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.434609505000001, 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29446849200000003, 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 208 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561411+0000", 2024-04-18T23:11:32.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9179999999999998e-06 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561421+0000", 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4599999999999992e-06 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561427+0000", 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561582+0000", 2024-04-18T23:11:32.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000154652 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561592+0000", 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.004e-05 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644937+0000", 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.083345591999999996 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644974+0000", 2024-04-18T23:11:32.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6835000000000002e-05 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855704+0000", 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.21072980599999999 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855787+0000", 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3442000000000005e-05 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855876+0000", 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8660999999999997e-05 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.021 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 42/31 ECSubWrite(tid=90, reqid=client.4568.0:279, at_version=0'0, trim_to=0'0, roll_forward_to=42'41))", 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:07.396027+0000", 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.599990103, 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20190517899999999, 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396027+0000", 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396027+0000", 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396029+0000", 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.553e-06 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396049+0000", 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0109e-05 2024-04-18T23:11:32.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396078+0000", 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9102e-05 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.396096+0000", 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8301999999999999e-05 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.596441+0000", 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20034464299999999 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.596477+0000", 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6189000000000002e-05 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.597862+0000", 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013854780000000001 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.597932+0000", 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9802999999999996e-05 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.025 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:331 3.8s0 3:1c19a06b:::smithi14230322-28:head [stat,delete] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.567129740999999, 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205262474, 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 331 2024-04-18T23:11:32.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428890+0000", 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4830000000000002e-06 2024-04-18T23:11:32.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428901+0000", 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0852999999999999e-05 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428933+0000", 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2234999999999999e-05 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428956+0000", 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3436000000000001e-05 2024-04-18T23:11:32.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.429116+0000", 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015961600000000001 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.429402+0000", 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028557099999999998 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.431719+0000", 2024-04-18T23:11:32.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002317117 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.434261+0000", 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025422980000000001 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.633920+0000", 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.199659052 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.634150+0000", 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022981299999999999 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 46/31 ECSubWriteReply(tid=21, last_complete=46'14, committed=1, applied=1))", 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:11.636567+0000", 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 20.359449606999998, 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19982181299999999, 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636567+0000", 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636567+0000", 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636569+0000", 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.403e-06 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636578+0000", 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8689999999999993e-06 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636605+0000", 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7056e-05 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.636619+0000", 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4269e-05 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.836343+0000", 2024-04-18T23:11:32.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19972390100000001 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.836389+0000", 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6315e-05 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.730926+0000", 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.265090621999999, 2024-04-18T23:11:32.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20030836399999999, 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730926+0000", 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730926+0000", 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730928+0000", 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.547e-06 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730940+0000", 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2079000000000001e-05 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730975+0000", 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5361999999999998e-05 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.730992+0000", 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6426000000000002e-05 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931197+0000", 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.200204826 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931235+0000", 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8124000000000003e-05 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.037 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.264942165000001, 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30085146299999999, 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731076+0000", 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6599999999999994e-07 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731085+0000", 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6670000000000002e-06 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731102+0000", 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7368999999999999e-05 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731113+0000", 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1552999999999999e-05 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031861+0000", 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30074756000000002 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031926+0000", 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5347999999999998e-05 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731177+0000", 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.264840529000001, 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20023347999999999, 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.041 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731177+0000", 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731177+0000", 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731178+0000", 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4959999999999999e-06 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731186+0000", 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3259999999999996e-06 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731210+0000", 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3938999999999999e-05 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731223+0000", 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.305e-05 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931361+0000", 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20013773200000001 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931410+0000", 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8937000000000003e-05 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.044 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731597+0000", 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.264420443999999, 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.199919018, 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731597+0000", 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731597+0000", 2024-04-18T23:11:32.045 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731598+0000", 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4470000000000001e-06 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731608+0000", 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5089999999999999e-06 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.046 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731638+0000", 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0574999999999998e-05 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731655+0000", 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6592999999999999e-05 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931474+0000", 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19981897600000001 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.047 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931516+0000", 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1918000000000001e-05 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731611+0000", 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.264406574999999, 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20007641100000001, 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.048 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731611+0000", 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731611+0000", 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.049 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731611+0000", 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0999999999999998e-07 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731625+0000", 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.366e-05 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731642+0000", 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7042e-05 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.050 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731658+0000", 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6543000000000001e-05 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931663+0000", 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20000464400000001 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931687+0000", 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3811999999999998e-05 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.051 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 46/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731775+0000", 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.264242124999999, 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19982924099999999, 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731775+0000", 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.052 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731775+0000", 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731776+0000", 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.486e-06 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731784+0000", 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7370000000000006e-06 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.053 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731802+0000", 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7492000000000001e-05 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731811+0000", 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.713e-06 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931577+0000", 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19976532799999999 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.054 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.931604+0000", 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7484999999999999e-05 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 56/31 ECSubWrite(tid=164, reqid=client.4568.0:431, at_version=56'73, trim_to=0'0, roll_forward_to=55'71))", 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:21.545022+0000", 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.4509949, 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102637434, 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.055 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.545022+0000", 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.545022+0000", 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.545024+0000", 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0059999999999999e-06 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.056 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.545114+0000", 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9660999999999994e-05 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.545154+0000", 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0125e-05 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.645270+0000", 2024-04-18T23:11:32.057 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10011627400000001 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.645433+0000", 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016226700000000001 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.645465+0000", 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2632000000000002e-05 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.647572+0000", 2024-04-18T23:11:32.058 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002106522 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.647660+0000", 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7947000000000003e-05 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.059 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.0s4 56/49 ECSubWrite(tid=95, reqid=client.4568.0:422, at_version=56'17, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:32.060 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:21.747013+0000", 2024-04-18T23:11:32.060 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.249003915999999, 2024-04-18T23:11:32.060 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102677452, 2024-04-18T23:11:32.060 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747013+0000", 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747013+0000", 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.061 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747015+0000", 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1660000000000001e-06 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747295+0000", 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027920400000000002 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747328+0000", 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3266000000000003e-05 2024-04-18T23:11:32.062 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747352+0000", 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4638999999999999e-05 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.847575+0000", 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100222123 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.847591+0000", 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6235999999999998e-05 2024-04-18T23:11:32.063 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.849642+0000", 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020509769999999998 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.849691+0000", 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8841000000000002e-05 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.064 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.0s4 56/49 ECSubWrite(tid=96, reqid=client.4568.0:423, at_version=56'18, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:21.747402+0000", 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.248615563, 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102849627, 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747402+0000", 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747402+0000", 2024-04-18T23:11:32.065 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747403+0000", 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.61e-06 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747475+0000", 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1973999999999999e-05 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747497+0000", 2024-04-18T23:11:32.066 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2345999999999999e-05 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747519+0000", 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1095999999999999e-05 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.848614+0000", 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10109589300000001 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.848631+0000", 2024-04-18T23:11:32.067 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6143000000000001e-05 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.850154+0000", 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015230280000000001 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.850251+0000", 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7536999999999998e-05 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.068 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.0s4 56/49 ECSubWrite(tid=97, reqid=client.4568.0:424, at_version=56'19, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:21.747556+0000", 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.248461352, 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10366125599999999, 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747556+0000", 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.069 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747556+0000", 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747557+0000", 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6700000000000002e-07 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747595+0000", 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8105000000000002e-05 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.070 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747618+0000", 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3336000000000002e-05 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.747637+0000", 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8732999999999998e-05 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.849289+0000", 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101652178 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.071 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.849322+0000", 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3265000000000001e-05 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.851126+0000", 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001803752 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:21.851217+0000", 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0920000000000004e-05 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.072 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 62/31 ECSubWrite(tid=191, reqid=client.4568.0:474, at_version=62'89, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.579289+0000", 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4167276050000002, 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.109061229, 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579289+0000", 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.073 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579289+0000", 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579291+0000", 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.517e-06 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579902+0000", 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00061076499999999998 2024-04-18T23:11:32.074 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579931+0000", 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9641999999999998e-05 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579961+0000", 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9753000000000001e-05 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680205+0000", 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100244103 2024-04-18T23:11:32.075 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680222+0000", 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6730000000000001e-05 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.688286+0000", 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0080643409999999992 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.688351+0000", 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4377999999999999e-05 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.076 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 66/31 ECSubWrite(tid=206, reqid=client.4568.0:507, at_version=66'102, trim_to=0'0, roll_forward_to=65'96))", 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:31.752252+0000", 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.24376467099999999, 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102278799, 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.752252+0000", 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.077 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.752252+0000", 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.752253+0000", 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3200000000000005e-07 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.752269+0000", 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5654999999999998e-05 2024-04-18T23:11:32.078 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.752281+0000", 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2627e-05 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.852372+0000", 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100090469 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.852552+0000", 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018034799999999999 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.852586+0000", 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3822999999999998e-05 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.854451+0000", 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001864814 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.854531+0000", 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0431000000000003e-05 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.080 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.081 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.081 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.081 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:32.081 INFO:tasks.rados.rados.0.smithi142.stdout:205: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-18T23:11:32.082 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:11:32.108 INFO:tasks.rados.rados.0.smithi142.stdout:206: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:32.118 INFO:tasks.rados.rados.0.smithi142.stdout:209: finishing copy_from racing read to smithi14230322-41 2024-04-18T23:11:32.127 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:32.127+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:32.147 INFO:tasks.rados.rados.0.smithi142.stdout:208: finishing write tid 4 to smithi14230322-37 2024-04-18T23:11:32.147 INFO:tasks.rados.rados.0.smithi142.stdout:208: oid 37 updating version 0 to 100 2024-04-18T23:11:32.147 INFO:tasks.rados.rados.0.smithi142.stdout:208: oid 37 updating version 100 to 101 2024-04-18T23:11:32.147 INFO:tasks.rados.rados.0.smithi142.stdout:208: oid 37 updating version 101 to 102 2024-04-18T23:11:32.147 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 37 v 102 (ObjNum 85 snap 20 seq_num 85) dirty exists 2024-04-18T23:11:32.147 INFO:tasks.rados.rados.0.smithi142.stdout:208: left oid 37 (ObjNum 85 snap 20 seq_num 85) 2024-04-18T23:11:32.195 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:32.194+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:32.251 INFO:tasks.rados.rados.0.smithi142.stdout:209: finishing copy_from to smithi14230322-41 2024-04-18T23:11:32.251 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 41 v 20 (ObjNum 77 snap 14 seq_num 77) dirty exists 2024-04-18T23:11:32.254 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:32.254 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:32.254 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:32.268 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:11:32.315 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:32.315+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:32.390 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:32.390 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:32.390 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:32.390 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:32.390 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:32.397 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:32.392+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:32.405 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:11:32.497 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:32.496+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:32.530 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T23:11:32.537 INFO:tasks.rados.rados.0.smithi142.stdout:204: done (6 left) 2024-04-18T23:11:32.537 INFO:tasks.rados.rados.0.smithi142.stdout:205: done (5 left) 2024-04-18T23:11:32.537 INFO:tasks.rados.rados.0.smithi142.stdout:206: done (4 left) 2024-04-18T23:11:32.537 INFO:tasks.rados.rados.0.smithi142.stdout:207: done (3 left) 2024-04-18T23:11:32.537 INFO:tasks.rados.rados.0.smithi142.stdout:208: done (2 left) 2024-04-18T23:11:32.537 INFO:tasks.rados.rados.0.smithi142.stdout:209: done (1 left) 2024-04-18T23:11:32.537 INFO:tasks.rados.rados.0.smithi142.stdout:210: done (0 left) 2024-04-18T23:11:32.537 INFO:tasks.rados.rados.0.smithi142.stdout:211: delete oid 31 current snap is 21 2024-04-18T23:11:32.580 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:32.580 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:32.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:32.580 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:32.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.580 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.1s2 33/31 ECSubWrite(tid=3, reqid=client.4568.0:2, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:32.580 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526871+0000", 2024-04-18T23:11:32.580 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.052789926000003, 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10188062000000001, 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526871+0000", 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526871+0000", 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526873+0000", 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526917+0000", 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4326999999999998e-05 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9663000000000003e-05 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526981+0000", 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3915999999999999e-05 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.627401+0000", 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10041953200000001 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.627425+0000", 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4085000000000001e-05 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628646+0000", 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012212010000000001 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628752+0000", 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010604599999999999 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.es1 36/31 ECSubRead(tid=16, to_read={3:7a4badf6:::smithi14230322-17:head=0,69632,0}, subchunks={3:7a4badf6:::smithi14230322-17:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.870730+0000", 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.708931188000001, 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20070310299999999, 2024-04-18T23:11:32.584 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870730+0000", 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870730+0000", 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870731+0000", 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4099999999999998e-07 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870743+0000", 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2034e-05 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870760+0000", 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6956000000000002e-05 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.870776+0000", 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6276000000000001e-05 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070936+0000", 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20016004100000001 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.071433+0000", 2024-04-18T23:11:32.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049705500000000004 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 42/31 ECSubWrite(tid=89, reqid=client.4568.0:279, at_version=42'41, trim_to=0'0, roll_forward_to=42'39))", 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:07.094046+0000", 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 25.485614797, 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103753032, 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.094046+0000", 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.094046+0000", 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.094048+0000", 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9630000000000001e-06 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.095150+0000", 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011015490000000001 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.095191+0000", 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1254999999999997e-05 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.195288+0000", 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100096488 2024-04-18T23:11:32.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.195417+0000", 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012939399999999999 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.195447+0000", 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9889000000000001e-05 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.197703+0000", 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022562319999999999 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.197799+0000", 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6261999999999994e-05 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 43/31 ECSubWrite(tid=91, reqid=client.4568.0:285, at_version=43'43, trim_to=0'0, roll_forward_to=42'41))", 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:07.691346+0000", 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.888315545000001, 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10290337400000001, 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691346+0000", 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691346+0000", 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691346+0000", 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8599999999999999e-07 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691450+0000", 2024-04-18T23:11:32.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000103819 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691478+0000", 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7742e-05 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.691495+0000", 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7102e-05 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.791721+0000", 2024-04-18T23:11:32.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10022610699999999 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.791749+0000", 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7948000000000001e-05 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.794166+0000", 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024171309999999999 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:07.794249+0000", 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3139000000000005e-05 2024-04-18T23:11:32.595 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 47/31 ECSubWrite(tid=116, reqid=client.4568.0:335, at_version=47'35, trim_to=0'0, roll_forward_to=47'33))", 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.936331+0000", 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.643329877999999, 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10251489, 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936331+0000", 2024-04-18T23:11:32.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936331+0000", 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936333+0000", 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0729999999999999e-06 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936461+0000", 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012804099999999999 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936493+0000", 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1216000000000001e-05 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036630+0000", 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10013704399999999 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036792+0000", 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016205599999999999 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036823+0000", 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1636e-05 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038757+0000", 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001933571 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038846+0000", 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9252999999999994e-05 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 47/31 ECSubWrite(tid=118, reqid=client.4568.0:348, at_version=47'57, trim_to=0'0, roll_forward_to=44'55))", 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:13.165615+0000", 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.414046591999998, 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10286374, 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165615+0000", 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165615+0000", 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165616+0000", 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.753e-06 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165941+0000", 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000324323 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.165970+0000", 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9572000000000001e-05 2024-04-18T23:11:32.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.266070+0000", 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010000300000001 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.266260+0000", 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019000500000000001 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.266277+0000", 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6957e-05 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.268410+0000", 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021329729999999998 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.268478+0000", 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8153999999999998e-05 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.6s6 47/31 ECSubWrite(tid=20, reqid=client.4568.0:346, at_version=47'14, trim_to=0'0, roll_forward_to=43'12))", 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:13.300001+0000", 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.279660581000002, 2024-04-18T23:11:32.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102818785, 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300001+0000", 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300001+0000", 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300001+0000", 2024-04-18T23:11:32.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3700000000000001e-07 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300296+0000", 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029461 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.300331+0000", 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5785999999999999e-05 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.400492+0000", 2024-04-18T23:11:32.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10016021999999999 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.400641+0000", 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000149667 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.400661+0000", 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9848999999999999e-05 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.402742+0000", 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002081009 2024-04-18T23:11:32.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.402819+0000", 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7206999999999994e-05 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 47/31 ECSubWrite(tid=121, reqid=client.4568.0:352, at_version=47'38, trim_to=0'0, roll_forward_to=47'36))", 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:13.371242+0000", 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.208418769000001, 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102859354, 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371242+0000", 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371242+0000", 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371244+0000", 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.35e-06 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371676+0000", 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043242600000000002 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371712+0000", 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5321000000000003e-05 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.371726+0000", 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4436999999999999e-05 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.471965+0000", 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10023947699999999 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.471982+0000", 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6269e-05 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.474035+0000", 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020528 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.474102+0000", 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7274000000000007e-05 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 53/31 ECSubWrite(tid=140, reqid=client.4568.0:402, at_version=53'61, trim_to=0'0, roll_forward_to=47'59))", 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:18.715934+0000", 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.863727710999999, 2024-04-18T23:11:32.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.104605728, 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.715934+0000", 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.715934+0000", 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.715934+0000", 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3399999999999998e-07 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.716420+0000", 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00048615799999999999 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.716450+0000", 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9615999999999999e-05 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.716470+0000", 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0055999999999999e-05 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.716669+0000", 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019932199999999999 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.716698+0000", 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8839e-05 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.820464+0000", 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10376592599999999 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:18.820539+0000", 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5077e-05 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 57/31 ECSubWrite(tid=166, reqid=osd.4.0:166, at_version=57'75, trim_to=0'0, roll_forward_to=56'73))", 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:22.531001+0000", 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.04866045, 2024-04-18T23:11:32.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10273341599999999, 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.531001+0000", 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.531001+0000", 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.531001+0000", 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.531068+0000", 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6351000000000002e-05 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.531109+0000", 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0951999999999997e-05 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631208+0000", 2024-04-18T23:11:32.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100099681 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631381+0000", 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017254799999999999 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631409+0000", 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8172999999999999e-05 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.633661+0000", 2024-04-18T23:11:32.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0022517570000000001 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.633734+0000", 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3233999999999997e-05 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 57/31 ECSubWrite(tid=167, reqid=osd.4.0:167, at_version=57'77, trim_to=0'0, roll_forward_to=56'73))", 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:22.631249+0000", 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.9484124099999995, 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20201448899999999, 2024-04-18T23:11:32.619 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.620 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.620 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.620 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631249+0000", 2024-04-18T23:11:32.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631249+0000", 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631250+0000", 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0559999999999999e-06 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631287+0000", 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7069e-05 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631308+0000", 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0723000000000001e-05 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.731401+0000", 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100093711 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.831671+0000", 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100269671 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.831687+0000", 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6283999999999999e-05 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.833165+0000", 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014771879999999999 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.833263+0000", 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8787000000000001e-05 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 57/31 ECSubWrite(tid=170, reqid=osd.4.0:167, at_version=0'0, trim_to=0'0, roll_forward_to=57'77))", 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:22.834221+0000", 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 9.7454401280000003, 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30088188700000001, 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834221+0000", 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834221+0000", 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834223+0000", 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.516e-06 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834241+0000", 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8712999999999999e-05 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834271+0000", 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9963000000000001e-05 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.934412+0000", 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10014097499999999 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.934586+0000", 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000173535 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.934601+0000", 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5574999999999999e-05 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:23.135038+0000", 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20043703500000001 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:23.135103+0000", 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4574999999999998e-05 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.627 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 61/31 ECSubWrite(tid=180, reqid=osd.4.0:180, at_version=61'64, trim_to=0'0, roll_forward_to=58'62))", 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.646462+0000", 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.9331990130000003, 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102682855, 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646462+0000", 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646462+0000", 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646463+0000", 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9299999999999997e-07 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646512+0000", 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9410999999999998e-05 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646538+0000", 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5823e-05 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646558+0000", 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9879999999999999e-05 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646703+0000", 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014523499999999999 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646732+0000", 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8639000000000001e-05 2024-04-18T23:11:32.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.749057+0000", 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102324954 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.749145+0000", 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8220000000000006e-05 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 61/31 ECSubWrite(tid=183, reqid=osd.4.0:183, at_version=61'66, trim_to=0'0, roll_forward_to=58'62))", 2024-04-18T23:11:32.631 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.647507+0000", 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.9321545990000004, 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102537431, 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647507+0000", 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647507+0000", 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647508+0000", 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6700000000000002e-07 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647550+0000", 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2792e-05 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647578+0000", 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7549e-05 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.647597+0000", 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8683000000000001e-05 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.748312+0000", 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10071538300000001 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.748329+0000", 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7492000000000001e-05 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.749965+0000", 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001635191 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.750044+0000", 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9374000000000003e-05 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 61/31 ECSubWrite(tid=44, reqid=osd.2.0:43, at_version=0'0, trim_to=0'0, roll_forward_to=61'31))", 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.245329+0000", 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3343323690000002, 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101975724, 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.635 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245329+0000", 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245329+0000", 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245332+0000", 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.717e-06 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245353+0000", 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.175e-05 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.245383+0000", 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9901999999999999e-05 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345525+0000", 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100141887 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345722+0000", 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000197061 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345745+0000", 2024-04-18T23:11:32.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3224999999999999e-05 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.347241+0000", 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014959540000000001 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.347305+0000", 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3227999999999998e-05 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.640 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 61/31 ECSubWrite(tid=190, reqid=osd.4.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:11:32.640 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.458871+0000", 2024-04-18T23:11:32.641 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.1207906269999999, 2024-04-18T23:11:32.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20139204499999999, 2024-04-18T23:11:32.641 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.641 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.641 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.641 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458871+0000", 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458871+0000", 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458872+0000", 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5740000000000001e-06 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458892+0000", 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0060000000000001e-05 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458921+0000", 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8714999999999999e-05 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458936+0000", 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5296000000000001e-05 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.643 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.559175+0000", 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100238271 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.559198+0000", 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3133e-05 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.660126+0000", 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100928437 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.644 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.660263+0000", 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013655899999999999 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 62/31 ECSubWrite(tid=191, reqid=client.4568.0:474, at_version=62'89, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.579357+0000", 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.0003039940000003, 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205588628, 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.645 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579357+0000", 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579357+0000", 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579358+0000", 2024-04-18T23:11:32.646 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.580046+0000", 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068822000000000002 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.580081+0000", 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4882000000000002e-05 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.647 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680226+0000", 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100145083 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680407+0000", 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000181068 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680437+0000", 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9983000000000001e-05 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.784862+0000", 2024-04-18T23:11:32.648 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.104425411 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.784946+0000", 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3608000000000003e-05 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 62/31 ECSubWrite(tid=192, reqid=client.4568.0:475, at_version=62'90, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.680295+0000", 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.8993662840000001, 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.104735655, 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.649 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680295+0000", 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680295+0000", 2024-04-18T23:11:32.650 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680297+0000", 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.866e-06 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680565+0000", 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026855299999999999 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.651 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680619+0000", 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3971000000000002e-05 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680652+0000", 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2953000000000001e-05 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.682603+0000", 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001950963 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.652 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.682623+0000", 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0092e-05 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.784958+0000", 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102334203 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.785031+0000", 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3053999999999995e-05 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.653 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 62/31 ECSubWrite(tid=193, reqid=client.4568.0:476, at_version=62'91, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.683043+0000", 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.8966177909999997, 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103186997, 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.683043+0000", 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.654 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.683043+0000", 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.683045+0000", 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8500000000000001e-06 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.683198+0000", 2024-04-18T23:11:32.655 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000152232 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.683237+0000", 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9567999999999998e-05 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.683266+0000", 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8473e-05 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.656 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.784069+0000", 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100803437 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.784086+0000", 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6537000000000001e-05 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.786164+0000", 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002078809 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.786230+0000", 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.6091000000000001e-05 2024-04-18T23:11:32.657 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 62/31 ECSubWrite(tid=194, reqid=client.4568.0:477, at_version=62'92, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.685341+0000", 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.8943200779999998, 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102034707, 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:32.658 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.685341+0000", 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.685341+0000", 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.685343+0000", 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.553e-06 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.685375+0000", 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2165999999999997e-05 2024-04-18T23:11:32.659 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.685401+0000", 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6542000000000001e-05 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.685417+0000", 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5523e-05 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.785124+0000", 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.099707552000000005 2024-04-18T23:11:32.660 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.785144+0000", 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9584000000000001e-05 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.787325+0000", 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002181306 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.787376+0000", 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0481e-05 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.661 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:32.662 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:32.662 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:32.663 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:11:32.664 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:32.662+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:32.764 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:32.763+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:32.789 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:32.789 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:32.789 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:32.789 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:32.803 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:11:32.864 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:32.860+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:32.966 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:32.962+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:32.971 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:32.985 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:11:33.066 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:33.068+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e23)", 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 50.725912147999999, 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50897375199999995, 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.105 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 1 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377585+0000", 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377587+0000", 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3089999999999998e-06 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377601+0000", 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3862999999999999e-05 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377623+0000", 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1741000000000001e-05 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.377644+0000", 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1024000000000001e-05 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.576350+0000", 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198706355 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:11:33.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.576365+0000", 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4748e-05 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.682700+0000", 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106335283 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.683098+0000", 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039820900000000001 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.683616+0000", 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051731399999999995 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.688337+0000", 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0047211509999999998 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886392+0000", 2024-04-18T23:11:33.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198055008 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886494+0000", 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000102203 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:42.886559+0000", 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4543999999999997e-05 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:110 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.509237161000001, 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41468469299999999, 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 110 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594260+0000", 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594261+0000", 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594266+0000", 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9690000000000002e-06 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594278+0000", 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2048999999999999e-05 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594286+0000", 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2900000000000002e-06 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.114 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.804556+0000", 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.21026968800000001 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.804784+0000", 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022826000000000001 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.805113+0000", 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00032918500000000001 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.908604+0000", 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103490558 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008799+0000", 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100195153 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008880+0000", 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.1329999999999996e-05 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.008945+0000", 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4400999999999995e-05 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:111 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.509199391000003, 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41616193699999998, 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.117 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 111 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594298+0000", 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6300000000000004e-07 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594305+0000", 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8299999999999998e-06 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594317+0000", 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1786e-05 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594327+0000", 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.7249999999999992e-06 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906203+0000", 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.31187656499999999 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906336+0000", 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013266800000000001 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.906542+0000", 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020588900000000001 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009954+0000", 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10341181100000001 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010185+0000", 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023115700000000001 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010324+0000", 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013910800000000001 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010460+0000", 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000135635 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.122 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:112 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.509158657999997, 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42114173300000002, 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 112 2024-04-18T23:11:33.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594338+0000", 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594339+0000", 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7700000000000004e-07 2024-04-18T23:11:33.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594484+0000", 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000144804 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594500+0000", 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5752000000000001e-05 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594511+0000", 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1039000000000001e-05 2024-04-18T23:11:33.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907178+0000", 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.31266760199999999 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907320+0000", 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014133099999999999 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.907545+0000", 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022546 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010139+0000", 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102593936 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015192+0000", 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0050530590000000004 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015364+0000", 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017223600000000001 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015480+0000", 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011573700000000001 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:113 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.508969860999997, 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42137598300000001, 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.128 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 113 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594527+0000", 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594528+0000", 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594533+0000", 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0130000000000001e-06 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594545+0000", 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2009e-05 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594553+0000", 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7389999999999999e-06 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009008+0000", 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41445484999999999 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009125+0000", 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000117773 2024-04-18T23:11:33.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.009350+0000", 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022510400000000001 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015618+0000", 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0062671189999999998 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015662+0000", 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.4619000000000001e-05 2024-04-18T23:11:33.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015798+0000", 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013600600000000001 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015903+0000", 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104941 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.133 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.508932541, 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62069844100000005, 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 114 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594565+0000", 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594570+0000", 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.186e-06 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594582+0000", 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1763000000000001e-05 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594590+0000", 2024-04-18T23:11:33.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5290000000000003e-06 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010572+0000", 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.415982252 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010727+0000", 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015558400000000001 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010971+0000", 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000243339 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015929+0000", 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049583639999999998 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215108+0000", 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19917934400000001 2024-04-18T23:11:33.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215199+0000", 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0556000000000005e-05 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215263+0000", 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3874000000000005e-05 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.508896214000004, 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62089894099999998, 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 115 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594746+0000", 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014446700000000001 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594761+0000", 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4792e-05 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594769+0000", 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6160000000000002e-06 2024-04-18T23:11:33.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013742+0000", 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.418972438 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013974+0000", 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023265699999999999 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.014328+0000", 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035345 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018069+0000", 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037411079999999999 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215346+0000", 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19727757900000001 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215435+0000", 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8245999999999995e-05 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215500+0000", 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4968000000000005e-05 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.508712963000001, 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.72133309899999998, 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.145 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 116 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594785+0000", 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594790+0000", 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4229999999999996e-06 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594803+0000", 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2196e-05 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594810+0000", 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6190000000000002e-06 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016062+0000", 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42125235 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016215+0000", 2024-04-18T23:11:33.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000152051 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016420+0000", 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000205916 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018082+0000", 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001661718 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.315968+0000", 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29788616600000001 2024-04-18T23:11:33.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316051+0000", 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2897000000000003e-05 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316117+0000", 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5866999999999998e-05 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 45.508675330000003, 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.721588585, 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 117 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594829+0000", 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7830000000000001e-06 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594841+0000", 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1505000000000001e-05 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.694940+0000", 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100099585 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016967+0000", 2024-04-18T23:11:33.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32202692799999999 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017224+0000", 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000256548 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017583+0000", 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000359689 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.020139+0000", 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025556649999999999 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316256+0000", 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29611688800000002 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316345+0000", 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9285999999999999e-05 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316410+0000", 2024-04-18T23:11:33.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5058e-05 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.028873941000001, 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.410819936, 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.156 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 140 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074623+0000", 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074624+0000", 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074804+0000", 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018036800000000001 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074824+0000", 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0018e-05 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074839+0000", 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4846e-05 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080014+0000", 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0051746780000000003 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080068+0000", 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4345000000000002e-05 2024-04-18T23:11:33.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080162+0000", 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4157e-05 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081144+0000", 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00098186100000000011 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485220+0000", 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40407592199999998 2024-04-18T23:11:33.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485314+0000", 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4161999999999997e-05 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485443+0000", 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012892900000000001 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:33.161 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.028640377999999, 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41085222500000002, 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 141 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:11:33.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074857+0000", 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074858+0000", 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0139999999999999e-06 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074863+0000", 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3759999999999999e-06 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074872+0000", 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3929999999999997e-06 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.074883+0000", 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0722e-05 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080730+0000", 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0058473249999999996 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080788+0000", 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7423999999999998e-05 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.080881+0000", 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2938999999999999e-05 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.081458+0000", 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00057760199999999998 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485586+0000", 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40412787700000002 2024-04-18T23:11:33.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485656+0000", 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0034000000000001e-05 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:49.485709+0000", 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2519000000000003e-05 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.166 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.969414833999998, 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50946670199999999, 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 162 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134083+0000", 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7000000000000003e-07 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134088+0000", 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2719999999999997e-06 2024-04-18T23:11:33.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134097+0000", 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1400000000000006e-06 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134104+0000", 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140241+0000", 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0061371919999999996 2024-04-18T23:11:33.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140319+0000", 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7768000000000005e-05 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140446+0000", 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126637 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243778+0000", 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103331863 2024-04-18T23:11:33.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643391+0000", 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.39961375599999999 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643476+0000", 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4274e-05 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643549+0000", 2024-04-18T23:11:33.171 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3071000000000006e-05 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.969381673999997, 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50963481200000005, 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.172 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 163 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.173 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134116+0000", 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134123+0000", 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4930000000000003e-06 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134131+0000", 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9490000000000002e-06 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.174 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134138+0000", 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8399999999999997e-06 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140848+0000", 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067093389999999999 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140969+0000", 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121787 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141164+0000", 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019482700000000001 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243782+0000", 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10261785900000001 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643626+0000", 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.399843858 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643699+0000", 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2657999999999995e-05 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643750+0000", 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1481999999999999e-05 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.969347583000001, 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50988570200000005, 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 164 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134150+0000", 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4000000000000001e-07 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134332+0000", 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018177199999999999 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134351+0000", 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9321999999999999e-05 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134368+0000", 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6994999999999999e-05 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141740+0000", 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073716420000000003 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141823+0000", 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3042999999999998e-05 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141951+0000", 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127976 2024-04-18T23:11:33.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243787+0000", 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101836063 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643874+0000", 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40008676199999998 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643964+0000", 2024-04-18T23:11:33.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0362e-05 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.644035+0000", 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1124999999999994e-05 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.969088347000003, 2024-04-18T23:11:33.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.54739116099999996, 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 165 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134410+0000", 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5770000000000001e-06 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134420+0000", 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0175e-05 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134439+0000", 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8125e-05 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134453+0000", 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4956e-05 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242648+0000", 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.108194819 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242783+0000", 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013431599999999999 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242992+0000", 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020903 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.244280+0000", 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012883669999999999 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681587+0000", 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.43730722300000002 2024-04-18T23:11:33.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681676+0000", 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8776000000000005e-05 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681800+0000", 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012379700000000001 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:11 3.2s0 3:42341748:::smithi14230322-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:33.188 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.585198296999998, 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50900182400000005, 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 11 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.189 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518300+0000", 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.823e-06 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518324+0000", 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3238999999999999e-05 2024-04-18T23:11:33.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518345+0000", 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0828000000000001e-05 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518359+0000", 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4912000000000001e-05 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521623+0000", 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003263808 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.522045+0000", 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00042140500000000002 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.523231+0000", 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011861840000000001 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530352+0000", 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071208340000000004 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.027150+0000", 2024-04-18T23:11:33.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49679807100000001 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.027300+0000", 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015071999999999999 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:25 3.2s0 3:4250b374:::smithi14230322-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.581618397, 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.51011498, 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.193 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 25 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:33.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521881+0000", 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4430000000000002e-06 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524055+0000", 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021740499999999999 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524086+0000", 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1414000000000001e-05 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524107+0000", 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0716e-05 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524323+0000", 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215675 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524696+0000", 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037315500000000001 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526737+0000", 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020405380000000002 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530355+0000", 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036182720000000001 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.031805+0000", 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50145000299999998 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.031994+0000", 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018871400000000001 2024-04-18T23:11:33.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:26 3.2s0 3:4250b374:::smithi14230322-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.579289304, 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60852922899999995, 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.198 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 26 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524210+0000", 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524220+0000", 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0056e-05 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524242+0000", 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2212e-05 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525734+0000", 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001491783 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528099+0000", 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023654380000000001 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528318+0000", 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000219065 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.529513+0000", 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011952530000000001 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636414+0000", 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106900906 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132457+0000", 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49604301899999997 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132737+0000", 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027950099999999998 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:31 3.2s0 3:42c34e84:::smithi14230322-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.577174880000001, 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60793644899999999, 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 31 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526324+0000", 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9400000000000001e-06 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526836+0000", 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051192200000000005 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526869+0000", 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2978000000000002e-05 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526890+0000", 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481999999999999e-05 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531167+0000", 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042762490000000002 2024-04-18T23:11:33.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531508+0000", 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034109400000000001 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.533620+0000", 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021122279999999998 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636418+0000", 2024-04-18T23:11:33.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102798509 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134075+0000", 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.497656453 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134259+0000", 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000183594 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:32 3.2s0 3:42c34e84:::smithi14230322-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:33.207 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.576571970000003, 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60874687999999999, 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 32 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526927+0000", 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0389999999999999e-06 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526936+0000", 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2599999999999994e-06 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:11:33.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0302e-05 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526973+0000", 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6359e-05 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634473+0000", 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.107499583 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634639+0000", 2024-04-18T23:11:33.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016687200000000001 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.635580+0000", 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094034999999999995 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636830+0000", 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012506220000000001 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135509+0000", 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49867835399999999 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:33.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:33.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:33.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135672+0000", 2024-04-18T23:11:33.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000163139 2024-04-18T23:11:33.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.212 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.212 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:33.212 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:33.212 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:33.213 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:11:33.214 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:33.210+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:33.315 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:33.315+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:11:33.344 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:33.358 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:11:33.416 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:33.414+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:33.476 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:33.477 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:33.477 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:33.477 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:33.490 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:11:33.503 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-04-18T23:11:33.516 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:33.515+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:33.618 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:33.615+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:33.640 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:33.640 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:114 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'76, mlcod=28'72)", 2024-04-18T23:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:48.011152+0000", 2024-04-18T23:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.647460614000003, 2024-04-18T23:11:33.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20343882299999999, 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011152+0000", 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011152+0000", 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011154+0000", 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.013e-06 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011169+0000", 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5665999999999999e-05 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011194+0000", 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4170999999999999e-05 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.011209+0000", 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5766000000000001e-05 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.111484+0000", 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100274822 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.111517+0000", 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2700999999999997e-05 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214524+0000", 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103007425 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:48.214591+0000", 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6259000000000004e-05 2024-04-18T23:11:33.666 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:133 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'91, mlcod=28'85)", 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.381190609000001, 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20553276400000001, 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277424+0000", 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.21e-06 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277436+0000", 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1693e-05 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277464+0000", 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8608000000000001e-05 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377595+0000", 2024-04-18T23:11:33.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100130684 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377730+0000", 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013455499999999999 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377781+0000", 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1406999999999998e-05 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:33.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482870+0000", 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105089222 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482955+0000", 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4384999999999999e-05 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:155 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'109, mlcod=28'105)", 2024-04-18T23:11:33.671 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.523652499000001, 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20593233399999999, 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134962+0000", 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8560000000000001e-06 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135074+0000", 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112357 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135099+0000", 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4705000000000001e-05 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135115+0000", 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5855999999999999e-05 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135494+0000", 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037909800000000002 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135551+0000", 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7176999999999998e-05 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340852+0000", 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20530157900000001 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340892+0000", 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9706000000000002e-05 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:156 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'110, mlcod=28'105)", 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.522845871999998, 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.205157487, 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.676 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135768+0000", 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2130000000000001e-06 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135779+0000", 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1749000000000001e-05 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135794+0000", 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453e-05 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135807+0000", 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2599000000000001e-05 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136859+0000", 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010521199999999999 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136911+0000", 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2096999999999999e-05 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340896+0000", 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20398571500000001 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340924+0000", 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7540999999999999e-05 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:157 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'111, mlcod=28'105)", 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:11:33.680 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.522131180000002, 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20611499799999999, 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136483+0000", 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.576e-06 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136499+0000", 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6529e-05 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136520+0000", 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0571999999999999e-05 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136532+0000", 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.223e-05 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237736+0000", 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10120357200000001 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237776+0000", 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0397e-05 2024-04-18T23:11:33.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342561+0000", 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104784615 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342596+0000", 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5506999999999997e-05 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:201 3.es0 3:78aee268:::smithi14230322-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.096856760999998, 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41057925299999998, 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561757+0000", 2024-04-18T23:11:33.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.47e-07 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561762+0000", 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.04e-06 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561777+0000", 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4962e-05 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561785+0000", 2024-04-18T23:11:33.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4500000000000004e-06 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562118+0000", 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000332882 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562139+0000", 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1101999999999999e-05 2024-04-18T23:11:33.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.769814+0000", 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.207675318 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770164+0000", 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034956199999999998 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.689 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.972335+0000", 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20217098999999999 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.4.21:1 3.es0 3:7a4badf6:::smithi14230322-17:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 32.897079927999997, 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31203379199999998, 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.690 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761535+0000", 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7999999999999999e-06 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761547+0000", 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1279e-05 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761568+0000", 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1549e-05 2024-04-18T23:11:33.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761588+0000", 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0426000000000001e-05 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770853+0000", 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092644660000000007 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.771105+0000", 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025160800000000001 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073566+0000", 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30246166400000002 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.694 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:333 3.es0 3:799dcc49:::smithi14230322-39:head [create,setxattr Yjk]k``VRNVkfm]Rnjieeerecjk`Wm (15648) in=30b,setxattr QgpqijWTrohYWOsYikii[YdsSee`QrtrZ]nSPTUt (1225) in=40b,setxattr l]iRdbVhahjiOeVf`Yc\\U_OYjY]nr[js\\NggsVVY (1015) in=40b,setxattr b^W[ZgXWiRmfON^`rihabiO]lNkro`P (15648) in=31b,setxattr VReWeRiYgNSZU\\d\\YOrNS`]ogtdOjVijWVeY`fhd (1051) in=40b,setxattr SYsfepqPdsc`cUg\\WaTfgrcUYhUbTiaQY_`rlt (15648) in=38b,setxattr P[`jheVP[fjpnXqdjPhddi_oWUf\\dWWbSlodc`eS (1086) in=40b,setxattr X[^VkT`QVNea_jfmlVkdafY]hrorgVhl]fqtiVie (1226) in=40b,setxattr Xs\\WjfX^_Ss_RPr_tT^a]fnSSlYrcPTbmSaksi_c (19437) in=40b,setxattr QUn_^p]__nRhn\\srj]jla^iQSPOpgQcVcQZU^ool (1823) in=40b,setxattr f\\^WXdjheR\\aQSYk]Un_^hkrZTSVdhiZqgcin[Sa (1155) in=40b,setxattr bkb]mNeZddhQrcsORPSgqbWecOTtcPYZaRQOg`nY (1614) in=40b,setxattr _T_atc\\\\Rml^aOPkNsbsb^VatOWSs\\URSjVqfe\\Z (1226) in=40b,setxattr obYqlk]hgNOQgXa^ZhalmPcXl\\lkOdNNiYlOfdhq (1612) in=40b,setxattr VsbnoacTrXVpb]PV]kVPYbQP_qefUdfOgk`qUaNa (1999) in=40b,setxattr ZqTlaiiPajkNT^ce\\W\\oklalkiT`\\\\nhjcgpZrUa (3297) in=40b,setxattr QfUmVl\\][PRg\\ajWQS`]Oe^Zlig]b]Rs_o]NsY[i (244) in=40b,setxattr R_Ucr`a\\j]Ri_YSp[gamgRSRd`]UYnPRspctZ`R] (4068) in=40b,setxattr aj`m[hpi__Rrg_Z[m^rVOVrVgk^cS_]piWV]lkRZ (947) in=40b,setxattr ^nWmd_TeciTqUOnWXn_jXch]heNac[WdcqdPbiT^ (1647) in=40b] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.031256566, 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30962595399999998, 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627359+0000", 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3400000000000002e-06 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627676+0000", 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000317144 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627710+0000", 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3855e-05 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627753+0000", 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3341999999999999e-05 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627926+0000", 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017248799999999999 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627946+0000", 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9868999999999999e-05 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729527+0000", 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101581587 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729747+0000", 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021955299999999999 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.733259+0000", 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035115820000000001 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739147+0000", 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058886169999999996 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936726+0000", 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.197578422 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936982+0000", 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025615499999999999 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:344 3.es0 3:7a4badf6:::smithi14230322-17:head [delete] snapc 8=[7,4] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:33.700 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.017198009000001, 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.295980827, 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 344 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:11:33.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641417+0000", 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2630000000000002e-06 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641427+0000", 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0674e-05 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641460+0000", 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2768999999999997e-05 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641482+0000", 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1959000000000001e-05 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641642+0000", 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016009100000000001 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641653+0000", 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.128e-05 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735170+0000", 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.093517063999999997 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735466+0000", 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029521400000000002 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.736750+0000", 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012846540000000001 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739533+0000", 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027831589999999999 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:33.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937242+0000", 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19770827199999999 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937395+0000", 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000153428 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 56/49 ECSubWrite(tid=94, reqid=client.4568.0:421, at_version=56'16, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:11:33.706 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.01522759, 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.27351094799999998, 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:11:33.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643386+0000", 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4300000000000002e-07 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643802+0000", 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000416136 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643934+0000", 2024-04-18T23:11:33.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013215300000000001 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643959+0000", 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4973e-05 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.812936+0000", 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.168977035 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.812952+0000", 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6419e-05 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916818+0000", 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10386601099999999 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916896+0000", 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7378000000000004e-05 2024-04-18T23:11:33.710 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 56/49 ECSubWrite(tid=95, reqid=client.4568.0:422, at_version=56'17, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.00961234, 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26796196300000003, 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649002+0000", 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8199999999999999e-06 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649359+0000", 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035705699999999998 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649390+0000", 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.146e-05 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749486+0000", 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100095925 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.814696+0000", 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.065209634000000002 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.814713+0000", 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7082999999999999e-05 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916904+0000", 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10219086500000001 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.714 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916962+0000", 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8118999999999997e-05 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.7.22:2 3.es0 3:799dcc49:::smithi14230322-39:head [assert-version v16,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.744250+0000", 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.914362564999999, 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.204506466, 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:33.715 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.744250+0000", 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.744250+0000", 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.744253+0000", 2024-04-18T23:11:33.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8200000000000001e-06 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.744264+0000", 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1503e-05 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.744285+0000", 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0639e-05 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.744303+0000", 2024-04-18T23:11:33.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8410000000000002e-05 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.744444+0000", 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014115300000000001 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.744605+0000", 2024-04-18T23:11:33.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016057199999999999 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.948756+0000", 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.204151369 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:465 3.es0 3:799dcc49:::smithi14230322-39:head [read 0~638976,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.553539+0000", 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 7.105073011, 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.204466171, 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 465 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.720 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553539+0000", 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553539+0000", 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553542+0000", 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.362e-06 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553552+0000", 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.048e-05 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553572+0000", 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0299e-05 2024-04-18T23:11:33.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553584+0000", 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1610999999999999e-05 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553654+0000", 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9496e-05 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553670+0000", 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6025999999999999e-05 2024-04-18T23:11:33.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.655117+0000", 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101447736 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.655251+0000", 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000134098 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.758006+0000", 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10275406300000001 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:466 3.es0 3:799dcc49:::smithi14230322-39:head [read 0~638976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.553608+0000", 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 7.1050046619999998, 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20502858400000001, 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:33.725 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 466 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553608+0000", 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553608+0000", 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553608+0000", 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7e-07 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553612+0000", 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8469999999999998e-06 2024-04-18T23:11:33.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553618+0000", 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0909999999999998e-06 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553623+0000", 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7389999999999999e-06 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553765+0000", 2024-04-18T23:11:33.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014253599999999999 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553786+0000", 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0999e-05 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.655768+0000", 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101981563 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.655911+0000", 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014265999999999999 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.758636+0000", 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102725679 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:467 3.es0 3:799dcc49:::smithi14230322-39:head [read 0~638976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:11:33.730 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 7.1049708799999998, 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39041841700000002, 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 467 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553642+0000", 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3000000000000001e-07 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553645+0000", 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.813e-06 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553650+0000", 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6840000000000001e-06 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553655+0000", 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654393+0000", 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100738487 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654401+0000", 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9340000000000006e-06 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.657013+0000", 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026115800000000001 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.657158+0000", 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014565499999999999 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.944060+0000", 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28690162600000002 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=42, last_complete=61'29, committed=1, applied=1))", 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.8991579659999998, 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28503450299999999, 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.735 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759455+0000", 2024-04-18T23:11:33.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7100000000000001e-07 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759459+0000", 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1760000000000003e-06 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759471+0000", 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1523e-05 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.859553+0000", 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10008174 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.044450+0000", 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18489760999999999 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.044489+0000", 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8683000000000002e-05 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.738 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=44, last_complete=61'31, committed=1, applied=1))", 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.2994095850000003, 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29027006799999999, 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359204+0000", 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.282e-06 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359210+0000", 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6579999999999999e-06 2024-04-18T23:11:33.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359227+0000", 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7283999999999999e-05 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359257+0000", 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9643000000000001e-05 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649393+0000", 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29013631200000001 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649473+0000", 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9888999999999994e-05 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=45, last_complete=61'33, committed=1, applied=1))", 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.2982783449999999, 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28925603500000002, 2024-04-18T23:11:33.742 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360335+0000", 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8999999999999995e-07 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360340+0000", 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4160000000000003e-06 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360354+0000", 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3709999999999999e-05 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360362+0000", 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6120000000000001e-06 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649559+0000", 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28919758400000001 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649590+0000", 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0822999999999999e-05 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.745 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=45, last_complete=61'33, committed=1, applied=1))", 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.548610+0000", 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 6.1100028289999999, 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202989485, 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:33.746 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.548610+0000", 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.548610+0000", 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.548610+0000", 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8100000000000001e-07 2024-04-18T23:11:33.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.548616+0000", 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7869999999999998e-06 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.548631+0000", 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4613999999999999e-05 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.648801+0000", 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10016976800000001 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.750470+0000", 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10166950299999999 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.751599+0000", 2024-04-18T23:11:33.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001128932 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:497 3.es0 3:799dcc49:::smithi14230322-39:head [rmxattr QgpqijWTrohYWOsYikii[YdsSee`QrtrZ]nSPTUt in=40b,rmxattr R_Ucr`a\\j]Ri_YSp[gamgRSRd`]UYnPRspctZ`R] in=40b,rmxattr VReWeRiYgNSZU\\d\\YOrNS`]ogtdOjVijWVeY`fhd in=40b,rmxattr X[^VkT`QVNea_jfmlVkdafY]hrorgVhl]fqtiVie in=40b,rmxattr _T_atc\\\\Rml^aOPkNsbsb^VatOWSs\\URSjVqfe\\Z in=40b,rmxattr bkb]mNeZddhQrcsORPSgqbWecOTtcPYZaRQOg`nY in=40b] snapc 15=[15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.127975406, 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21045935599999999, 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:33.750 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 497 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530641+0000", 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7869999999999999e-06 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530672+0000", 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1642e-05 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530707+0000", 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4566000000000001e-05 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530726+0000", 2024-04-18T23:11:33.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9130000000000001e-05 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530864+0000", 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138307 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530878+0000", 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.379e-05 2024-04-18T23:11:33.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.534931+0000", 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004053035 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.535122+0000", 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190612 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.536668+0000", 2024-04-18T23:11:33.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015465800000000001 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.640407+0000", 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103738334 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.740945+0000", 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100538086 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.741096+0000", 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015148700000000001 2024-04-18T23:11:33.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.757 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.757 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.757 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:33.757 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:33.757 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:33.758 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:33.758 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:33.759 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:33.759 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:33.760 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:11:33.760 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:33.761 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:33.761 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:33.761 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:33.762 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:33.762 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:33.754+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:33.765 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:33.765 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:33.787 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:33.787 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:33.807 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:33.807 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:33.829 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:33.829 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:33.848 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:33.848 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:33.859 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:33.858+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:33.870 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:33.870 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:33.887 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:33.890 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:33.890 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:33.902 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:11:33.912 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:33.912 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:33.931 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:33.932 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:33.954 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:33.954 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:33.960 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:33.955+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:34.060 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:34.058+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:34.066 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:34.066 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:34.081 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:11:34.161 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:34.163+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:34.204 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:34.204 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:34.218 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:11:34.262 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:34.258+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:34.340 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:34.340 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:34.354 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:11:34.363 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:34.358+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:34.464 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:34.463+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:34.477 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:34.477 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:34.493 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:11:34.565 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:34.563+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:34.612 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:34.612 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:34.626 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:11:34.666 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:34.663+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:34.736 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:34.737 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:34.749 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:11:34.767 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:34.767+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:34.868 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:34.867+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:34.871 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:34.871 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:34.886 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:11:34.969 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:34.966+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:35.009 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:35.009 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:35.023 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:11:35.069 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:35.066+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:35.126 INFO:tasks.rados.rados.0.smithi142.stdout:211: done (0 left) 2024-04-18T23:11:35.126 INFO:tasks.rados.rados.0.smithi142.stdout:212: delete oid 26 current snap is 21 2024-04-18T23:11:35.157 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:35.157 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:35.170 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:35.170+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:35.171 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:11:35.272 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:35.270+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:35.294 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:35.294 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:35.307 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:11:35.373 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:35.371+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:35.430 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:35.430 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:35.432 INFO:tasks.rados.rados.0.smithi142.stdout:212: done (0 left) 2024-04-18T23:11:35.432 INFO:tasks.rados.rados.0.smithi142.stdout:213: copy_from oid 20 from oid 42 current snap is 21 2024-04-18T23:11:35.432 INFO:tasks.rados.rados.0.smithi142.stdout:214: delete oid 5 current snap is 21 2024-04-18T23:11:35.447 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:11:35.474 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:35.471+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:35.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T23:11:35.537 INFO:tasks.rados.rados.0.smithi142.stdout:214: done (1 left) 2024-04-18T23:11:35.537 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 13 current snap is 21 2024-04-18T23:11:35.537 INFO:tasks.rados.rados.0.smithi142.stdout:215: seq_num 89 ranges {1572864=737280,2310144=573440,2883584=196608} 2024-04-18T23:11:35.567 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:35.567 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:35.575 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:35.571+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:35.575 INFO:tasks.rados.rados.0.smithi142.stdout:215: writing smithi14230322-13 from 1572864 to 2310144 tid 1 2024-04-18T23:11:35.582 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:11:35.584 INFO:tasks.rados.rados.0.smithi142.stdout:215: writing smithi14230322-13 from 2310144 to 2883584 tid 2 2024-04-18T23:11:35.588 INFO:tasks.rados.rados.0.smithi142.stdout:215: writing smithi14230322-13 from 2883584 to 3080192 tid 3 2024-04-18T23:11:35.588 INFO:tasks.rados.rados.0.smithi142.stdout:216: read oid 19 snap -1 2024-04-18T23:11:35.588 INFO:tasks.rados.rados.0.smithi142.stdout:216: expect deleted 2024-04-18T23:11:35.588 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 21 current snap is 21 2024-04-18T23:11:35.588 INFO:tasks.rados.rados.0.smithi142.stdout:217: seq_num 90 ranges {475136=753664,1228800=720896,1949696=49152} 2024-04-18T23:11:35.603 INFO:tasks.rados.rados.0.smithi142.stdout:217: writing smithi14230322-21 from 475136 to 1228800 tid 1 2024-04-18T23:11:35.611 INFO:tasks.rados.rados.0.smithi142.stdout:217: writing smithi14230322-21 from 1228800 to 1949696 tid 2 2024-04-18T23:11:35.611 INFO:tasks.rados.rados.0.smithi142.stdout:217: writing smithi14230322-21 from 1949696 to 1998848 tid 3 2024-04-18T23:11:35.611 INFO:tasks.rados.rados.0.smithi142.stdout:218: delete oid 22 current snap is 21 2024-04-18T23:11:35.675 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:35.675+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:35.705 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:35.705 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:35.720 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:11:35.776 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:35.775+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:35.818 INFO:tasks.rados.rados.0.smithi142.stdout:216: done (4 left) 2024-04-18T23:11:35.818 INFO:tasks.rados.rados.0.smithi142.stdout:218: done (3 left) 2024-04-18T23:11:35.818 INFO:tasks.rados.rados.0.smithi142.stdout:219: read oid 43 snap -1 2024-04-18T23:11:35.818 INFO:tasks.rados.rados.0.smithi142.stdout:219: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-18T23:11:35.818 INFO:tasks.rados.rados.0.smithi142.stdout:220: copy_from oid 48 from oid 11 current snap is 21 2024-04-18T23:11:35.818 INFO:tasks.rados.rados.0.smithi142.stdout:221: copy_from oid 10 from oid 47 current snap is 21 2024-04-18T23:11:35.818 INFO:tasks.rados.rados.0.smithi142.stdout:222: read oid 29 snap 20 2024-04-18T23:11:35.818 INFO:tasks.rados.rados.0.smithi142.stdout:222: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-18T23:11:35.819 INFO:tasks.rados.rados.0.smithi142.stdout:223: delete oid 22 current snap is 21 2024-04-18T23:11:35.877 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:35.874+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:35.888 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:35.888 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:35.902 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:11:35.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:35.978 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:35.975+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:35.985 INFO:tasks.rados.rados.0.smithi142.stdout:215: finishing write tid 1 to smithi14230322-13 2024-04-18T23:11:35.985 INFO:tasks.rados.rados.0.smithi142.stdout:215: finishing write tid 2 to smithi14230322-13 2024-04-18T23:11:35.986 INFO:tasks.rados.rados.0.smithi142.stdout:215: finishing write tid 3 to smithi14230322-13 2024-04-18T23:11:36.019 INFO:tasks.rados.rados.0.smithi142.stdout:213: finishing copy_from racing read to smithi14230322-20 2024-04-18T23:11:36.027 INFO:tasks.rados.rados.0.smithi142.stdout:221: finishing copy_from racing read to smithi14230322-10 2024-04-18T23:11:36.028 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:36.028 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:36.031 INFO:tasks.rados.rados.0.smithi142.stdout:220: finishing copy_from racing read to smithi14230322-48 2024-04-18T23:11:36.044 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:11:36.079 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:36.079+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:36.086 INFO:tasks.rados.rados.0.smithi142.stdout:215: finishing write tid 4 to smithi14230322-13 2024-04-18T23:11:36.124 INFO:tasks.rados.rados.0.smithi142.stdout:217: finishing write tid 1 to smithi14230322-21 2024-04-18T23:11:36.125 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:36.125 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:36.146 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:36.147 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:36.167 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:36.168 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:36.180 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:36.178+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:36.182 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:11:36.190 INFO:tasks.rados.rados.0.smithi142.stdout:215: finishing write tid 5 to smithi14230322-13 2024-04-18T23:11:36.190 INFO:tasks.rados.rados.0.smithi142.stdout:215: oid 13 updating version 0 to 39 2024-04-18T23:11:36.190 INFO:tasks.rados.rados.0.smithi142.stdout:215: oid 13 updating version 39 to 40 2024-04-18T23:11:36.190 INFO:tasks.rados.rados.0.smithi142.stdout:215: oid 13 updating version 40 to 42 2024-04-18T23:11:36.190 INFO:tasks.rados.rados.0.smithi142.stdout:215: oid 13 version 42 is already newer than 41 2024-04-18T23:11:36.190 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 13 v 42 (ObjNum 89 snap 21 seq_num 89) dirty exists 2024-04-18T23:11:36.191 INFO:tasks.rados.rados.0.smithi142.stdout:215: left oid 13 (ObjNum 89 snap 21 seq_num 89) 2024-04-18T23:11:36.210 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:36.210 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:36.223 INFO:tasks.rados.rados.0.smithi142.stdout:217: finishing write tid 2 to smithi14230322-21 2024-04-18T23:11:36.223 INFO:tasks.rados.rados.0.smithi142.stdout:217: finishing write tid 3 to smithi14230322-21 2024-04-18T23:11:36.223 INFO:tasks.rados.rados.0.smithi142.stdout:217: finishing write tid 4 to smithi14230322-21 2024-04-18T23:11:36.224 INFO:tasks.rados.rados.0.smithi142.stdout:219: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-18T23:11:36.234 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:36.234 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:36.257 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:36.258 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:36.281 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:36.282 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:36.275+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:36.282 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:36.305 INFO:tasks.rados.rados.0.smithi142.stdout:213: finishing copy_from to smithi14230322-20 2024-04-18T23:11:36.305 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 20 v 12 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-18T23:11:36.305 INFO:tasks.rados.rados.0.smithi142.stdout:217: finishing write tid 5 to smithi14230322-21 2024-04-18T23:11:36.306 INFO:tasks.rados.rados.0.smithi142.stdout:217: oid 21 updating version 0 to 31 2024-04-18T23:11:36.306 INFO:tasks.rados.rados.0.smithi142.stdout:217: oid 21 version 31 is already newer than 28 2024-04-18T23:11:36.306 INFO:tasks.rados.rados.0.smithi142.stdout:217: oid 21 version 31 is already newer than 29 2024-04-18T23:11:36.306 INFO:tasks.rados.rados.0.smithi142.stdout:217: oid 21 version 31 is already newer than 30 2024-04-18T23:11:36.306 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 21 v 31 (ObjNum 90 snap 21 seq_num 90) dirty exists 2024-04-18T23:11:36.306 INFO:tasks.rados.rados.0.smithi142.stdout:217: left oid 21 (ObjNum 90 snap 21 seq_num 90) 2024-04-18T23:11:36.306 INFO:tasks.rados.rados.0.smithi142.stdout:221: finishing copy_from to smithi14230322-10 2024-04-18T23:11:36.306 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 10 v 14 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-18T23:11:36.306 INFO:tasks.rados.rados.0.smithi142.stdout:222: expect (ObjNum 28 snap 0 seq_num 28) 2024-04-18T23:11:36.307 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:36.307 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:36.313 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:36.313 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:213: done (7 left) 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:215: done (6 left) 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:217: done (5 left) 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:219: done (4 left) 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:221: done (3 left) 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:222: done (2 left) 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:223: done (1 left) 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:224: rollback oid 42 current snap is 21 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 42 to 10 2024-04-18T23:11:36.325 INFO:tasks.rados.rados.0.smithi142.stdout:225: delete oid 7 current snap is 21 2024-04-18T23:11:36.327 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:36.328 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:36.329 INFO:tasks.rados.rados.0.smithi142.stdout:220: finishing copy_from to smithi14230322-48 2024-04-18T23:11:36.329 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 48 v 6 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-18T23:11:36.329 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:11:36.382 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:36.378+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:36.443 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:36.443 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:36.449 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:36.449 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:36.449 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:11:36.449 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:36.464 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:11:36.471 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:36.482 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:36.478+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:36.498 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:36.498 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:36.523 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:36.530 INFO:tasks.rados.rados.0.smithi142.stdout:220: done (2 left) 2024-04-18T23:11:36.530 INFO:tasks.rados.rados.0.smithi142.stdout:225: done (1 left) 2024-04-18T23:11:36.530 INFO:tasks.rados.rados.0.smithi142.stdout:226: copy_from oid 40 from oid 8 current snap is 21 2024-04-18T23:11:36.531 INFO:tasks.rados.rados.0.smithi142.stdout:227: rollback oid 33 current snap is 21 2024-04-18T23:11:36.531 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 33 to 17 2024-04-18T23:11:36.531 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 13 current snap is 21 2024-04-18T23:11:36.531 INFO:tasks.rados.rados.0.smithi142.stdout:228: seq_num 94 ranges {3080192=425984,3506176=786432,4292608=376832} 2024-04-18T23:11:36.542 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-04-18T23:11:36.553 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:36.553 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:36.559 INFO:tasks.rados.rados.0.smithi142.stdout:228: writing smithi14230322-13 from 3080192 to 3506176 tid 1 2024-04-18T23:11:36.566 INFO:tasks.rados.rados.0.smithi142.stdout:228: writing smithi14230322-13 from 3506176 to 4292608 tid 2 2024-04-18T23:11:36.570 INFO:tasks.rados.rados.0.smithi142.stdout:228: writing smithi14230322-13 from 4292608 to 4669440 tid 3 2024-04-18T23:11:36.570 INFO:tasks.rados.rados.0.smithi142.stdout:229: rollback oid 28 current snap is 21 2024-04-18T23:11:36.570 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 28 to 18 2024-04-18T23:11:36.570 INFO:tasks.rados.rados.0.smithi142.stdout:224: finishing rollback tid 0 to smithi14230322-42 2024-04-18T23:11:36.570 INFO:tasks.rados.rados.0.smithi142.stdout:230: snap_remove snap 20 2024-04-18T23:11:36.576 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:36.576 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:36.582 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:36.578+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:36.587 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:11:36.600 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:36.600 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:36.600 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:36.600 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:36.600 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:36.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:11:36.652 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:36.653 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:36.700 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:36.698+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:36.716 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:36.716 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:36.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-18T23:11:36.773 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:36.773 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 33/31 ECSubWrite(tid=1, reqid=client.4568.0:16, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 40.264220905000002, 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104807236, 2024-04-18T23:11:36.783 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517773+0000", 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6599999999999996e-07 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517909+0000", 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013603000000000001 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.517960+0000", 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1122000000000003e-05 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.618057+0000", 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10009630999999999 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.618206+0000", 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000149259 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.618219+0000", 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2761e-05 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.622544+0000", 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043254590000000002 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.622580+0000", 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5729000000000003e-05 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.5s5 40/31 ECSubWrite(tid=27, reqid=osd.15.0:26, at_version=0'0, trim_to=0'0, roll_forward_to=40'15))", 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:04.973085+0000", 2024-04-18T23:11:36.787 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.808908206000002, 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.11391169600000001, 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973085+0000", 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973085+0000", 2024-04-18T23:11:36.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973086+0000", 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2099999999999995e-07 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973103+0000", 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7042999999999999e-05 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973153+0000", 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.003e-05 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:04.973166+0000", 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2340999999999999e-05 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086086+0000", 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.112919777 2024-04-18T23:11:36.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086100+0000", 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4949999999999999e-05 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086924+0000", 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00082378300000000002 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.086997+0000", 2024-04-18T23:11:36.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2850999999999998e-05 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.8s1 40/31 ECSubWrite(tid=19, reqid=osd.5.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:05.187735+0000", 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.594258911000001, 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201244059, 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187735+0000", 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187735+0000", 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187741+0000", 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4200000000000004e-06 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187762+0000", 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1095999999999999e-05 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.187791+0000", 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8875999999999999e-05 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.287866+0000", 2024-04-18T23:11:36.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100074872 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388009+0000", 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100142971 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388016+0000", 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5919999999999997e-06 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388929+0000", 2024-04-18T23:11:36.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00091366799999999997 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:05.388979+0000", 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9564000000000002e-05 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s3 42/31 ECSubWrite(tid=89, reqid=client.4568.0:279, at_version=42'41, trim_to=0'0, roll_forward_to=42'39))", 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:07.091573+0000", 2024-04-18T23:11:36.796 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.690420953, 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20162514400000001, 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091573+0000", 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091573+0000", 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091575+0000", 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0229999999999999e-06 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091804+0000", 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00022908800000000001 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091827+0000", 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3210999999999999e-05 2024-04-18T23:11:36.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091839+0000", 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2091e-05 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.292067+0000", 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20022828100000001 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.292074+0000", 2024-04-18T23:11:36.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8379999999999996e-06 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.293170+0000", 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010957250000000001 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.293198+0000", 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7886999999999999e-05 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.800 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs6 51/31 ECSubWrite(tid=24, reqid=client.4568.0:385, at_version=51'5, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.776073+0000", 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 20.005921074, 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.203822843, 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776073+0000", 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776073+0000", 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776074+0000", 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6899999999999999e-06 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776440+0000", 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036623500000000002 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.776490+0000", 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9808999999999997e-05 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876611+0000", 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100120291 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.977075+0000", 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100464015 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.977108+0000", 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3593000000000002e-05 2024-04-18T23:11:36.803 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.979820+0000", 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0027116419999999998 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.979895+0000", 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5568000000000006e-05 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.804 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs6 51/31 ECSubWrite(tid=25, reqid=client.4568.0:386, at_version=51'6, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.876665+0000", 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.90532842, 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.1052046, 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876665+0000", 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876665+0000", 2024-04-18T23:11:36.805 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876667+0000", 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4360000000000001e-06 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876763+0000", 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6008000000000006e-05 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876795+0000", 2024-04-18T23:11:36.806 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2045999999999998e-05 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876831+0000", 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6579000000000003e-05 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.978579+0000", 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101747526 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.978598+0000", 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9477e-05 2024-04-18T23:11:36.807 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981785+0000", 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00318651 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981870+0000", 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5018000000000005e-05 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs6 51/31 ECSubWrite(tid=26, reqid=client.4568.0:387, at_version=51'7, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.876865+0000", 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.905128404999999, 2024-04-18T23:11:36.808 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.105074846, 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876865+0000", 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876865+0000", 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.809 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876866+0000", 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2600000000000001e-07 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876899+0000", 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3246e-05 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876923+0000", 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3563000000000001e-05 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.810 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.876941+0000", 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8023e-05 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.979369+0000", 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10242828599999999 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.979399+0000", 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9759000000000001e-05 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.811 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981877+0000", 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002478175 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981940+0000", 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2867999999999995e-05 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds4 51/31 ECSubWrite(tid=50, reqid=client.4568.0:389, at_version=51'31, trim_to=0'0, roll_forward_to=47'29))", 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.877090+0000", 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.904904084999998, 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104646974, 2024-04-18T23:11:36.812 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877090+0000", 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877090+0000", 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877090+0000", 2024-04-18T23:11:36.813 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5400000000000001e-07 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877352+0000", 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00026140700000000001 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877379+0000", 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7902000000000002e-05 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.877401+0000", 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1440000000000001e-05 2024-04-18T23:11:36.814 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.977596+0000", 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10019465900000001 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.977611+0000", 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5583e-05 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981660+0000", 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040493580000000003 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.815 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.981737+0000", 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6070999999999997e-05 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:416 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~524288] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.517140+0000", 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.264853304000001, 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.21039138299999999, 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.816 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 416 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517140+0000", 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.817 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517140+0000", 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517143+0000", 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5900000000000002e-06 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517802+0000", 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000659215 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.818 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517830+0000", 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7472999999999999e-05 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.517843+0000", 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3261000000000001e-05 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618107+0000", 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100263651 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.819 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618120+0000", 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3016999999999999e-05 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.619287+0000", 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011671710000000001 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.619457+0000", 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016981799999999999 2024-04-18T23:11:36.820 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.621037+0000", 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001580084 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727045+0000", 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10600874 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727341+0000", 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029538900000000001 2024-04-18T23:11:36.821 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727532+0000", 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000190974 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:417 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~524288 in=524288b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.261890736, 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30835631899999999, 2024-04-18T23:11:36.822 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 417 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.823 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520104+0000", 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2389999999999999e-06 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520670+0000", 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00056593599999999996 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.824 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520690+0000", 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9945e-05 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520699+0000", 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0429999999999996e-06 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618217+0000", 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097518176999999998 2024-04-18T23:11:36.825 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618227+0000", 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7860000000000008e-06 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622364+0000", 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041371389999999997 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622653+0000", 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028858600000000002 2024-04-18T23:11:36.826 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.624345+0000", 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001691782 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727165+0000", 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102820087 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828093+0000", 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100928079 2024-04-18T23:11:36.827 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828459+0000", 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036652000000000001 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:418 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~245760 in=245760b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.260407880000001, 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30828365299999999, 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.828 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 418 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.829 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521587+0000", 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2100000000000001e-06 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521960+0000", 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000373402 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.830 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521972+0000", 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.207e-05 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521979+0000", 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7839999999999998e-06 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725251+0000", 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20327163200000001 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.831 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725428+0000", 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000177071 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.726423+0000", 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00099555200000000003 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828646+0000", 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102223007 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:36.832 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829746+0000", 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010997509999999999 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829869+0000", 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012317400000000001 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:419 3.6s0 3:635103ff:::smithi14230322-31:head [setxattr _header (57) in=7b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.260002863, 2024-04-18T23:11:36.833 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71019062499999996, 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 419 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.834 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2200000000000004e-07 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521994+0000", 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8289999999999999e-06 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.835 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521999+0000", 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8470000000000003e-06 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522003+0000", 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1250000000000003e-06 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828718+0000", 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30671521899999998 2024-04-18T23:11:36.836 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828796+0000", 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7502000000000004e-05 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829209+0000", 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041271100000000001 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829944+0000", 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00073506600000000004 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.837 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.231940+0000", 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.401996146 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232181+0000", 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024165800000000001 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:420 3.6s0 3:635103ff:::smithi14230322-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:36.838 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.25998519, 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.108457993, 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 420 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.839 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522009+0000", 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4999999999999998e-07 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522012+0000", 2024-04-18T23:11:36.840 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.819e-06 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522017+0000", 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2100000000000001e-06 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522021+0000", 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1400000000000002e-06 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829499+0000", 2024-04-18T23:11:36.841 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30747766399999998 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829621+0000", 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000122502 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232655+0000", 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.403033435 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232821+0000", 2024-04-18T23:11:36.842 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000166279 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.630466+0000", 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39764559399999999 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=25, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.627081+0000", 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.154912252000001, 2024-04-18T23:11:36.843 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20139538000000001, 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627081+0000", 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627081+0000", 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.844 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627083+0000", 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.443e-06 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627091+0000", 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0469999999999994e-06 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627114+0000", 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2679000000000001e-05 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.845 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.627125+0000", 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1513e-05 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.827940+0000", 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20081492200000001 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828477+0000", 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00053677599999999998 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.846 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=27, last_complete=56'20, committed=1, applied=1))", 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.031423+0000", 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.750570858, 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20115857000000001, 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.031423+0000", 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.847 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.031423+0000", 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.031424+0000", 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3200000000000001e-06 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.031432+0000", 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2910000000000007e-06 2024-04-18T23:11:36.848 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.031450+0000", 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8071999999999999e-05 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.131618+0000", 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100167955 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.849 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.231838+0000", 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100219266 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232581+0000", 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00074366600000000003 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.6s0 56/49 ECSubWriteReply(tid=28, last_complete=56'20, committed=1, applied=1))", 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.434262+0000", 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 14.347731418, 2024-04-18T23:11:36.850 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19502919099999999, 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.434262+0000", 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.434262+0000", 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.851 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.434264+0000", 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3740000000000001e-06 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.434273+0000", 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2799999999999992e-06 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.852 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.434289+0000", 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6300999999999999e-05 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.534361+0000", 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10007152800000001 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629143+0000", 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.094782535000000001 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.853 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629291+0000", 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014817300000000001 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs2 61/31 ECSubWrite(tid=41, reqid=osd.11.0:41, at_version=61'14, trim_to=0'0, roll_forward_to=60'10))", 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.654100+0000", 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.127893389, 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.202335298, 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.854 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654100+0000", 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654100+0000", 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654101+0000", 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1699999999999997e-07 2024-04-18T23:11:36.855 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654121+0000", 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0520999999999999e-05 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654137+0000", 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5160999999999999e-05 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.754240+0000", 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100103758 2024-04-18T23:11:36.856 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.854382+0000", 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100141194 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.854392+0000", 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0137e-05 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.856388+0000", 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001996421 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.857 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.856436+0000", 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7389000000000003e-05 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.5s5 61/49 ECSubWrite(tid=66, reqid=client.4568.0:469, at_version=61'38, trim_to=0'0, roll_forward_to=54'34))", 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.957126+0000", 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.8248675030000001, 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10640553, 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.858 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957126+0000", 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957126+0000", 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957128+0000", 2024-04-18T23:11:36.859 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3360000000000002e-06 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957676+0000", 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00054796300000000005 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957720+0000", 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4022e-05 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957747+0000", 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.667e-05 2024-04-18T23:11:36.860 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.057985+0000", 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100237558 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.058000+0000", 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5586e-05 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.063445+0000", 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.005444651 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.861 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.063532+0000", 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6743999999999995e-05 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.7.22:3 3.6s0 3:635103ff:::smithi14230322-31:head [assert-version v20,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:31.831327+0000", 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.9506667320000002, 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.21208626, 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.862 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831327+0000", 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831327+0000", 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831329+0000", 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2929999999999999e-06 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831339+0000", 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831359+0000", 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9743000000000001e-05 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831373+0000", 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4583000000000001e-05 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.935084+0000", 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103711043 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.935260+0000", 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017573299999999999 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:32.043413+0000", 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.108153165 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.865 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.5s5 69/68 ECSubRead(tid=75, to_read={3:a09a6894:::smithi14230322-25:e=0,262144,0}, subchunks={3:a09a6894:::smithi14230322-25:e=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.239669+0000", 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.54232422700000005, 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20126797900000001, 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.239669+0000", 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.866 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.239669+0000", 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.239670+0000", 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7300000000000005e-07 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.239679+0000", 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5320000000000006e-06 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.867 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.239697+0000", 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8374000000000001e-05 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.339780+0000", 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008233699999999 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.439972+0000", 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100192854 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:36.868 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:36.869 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.440937+0000", 2024-04-18T23:11:36.869 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00096500899999999996 2024-04-18T23:11:36.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.869 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.869 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:36.869 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:36.869 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:36.871 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:11:36.872 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:36.870+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:36.972 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:36.970+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:37.029 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:37.029 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:37.029 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:11:37.029 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:37.043 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:11:37.088 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:37.086+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:37.168 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:37.168 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:37.168 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:37.168 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:37.168 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:37.173 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:37.167+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:37.181 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:11:37.274 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:37.270+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:37.351 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:37.351 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:37.351 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:37.351 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:37.351 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.351 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:184 3.bs0 3:dd01ba5f:::smithi14230322-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.790002659999999, 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61588253999999998, 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 184 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.352 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8700000000000001e-07 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.353 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557493+0000", 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9149999999999999e-06 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557499+0000", 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2650000000000002e-06 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557504+0000", 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5689999999999998e-06 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.354 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654492+0000", 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.096987633000000004 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654498+0000", 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5860000000000001e-06 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757767+0000", 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103269131 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.355 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757839+0000", 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1878999999999994e-05 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967626+0000", 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.209786891 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967877+0000", 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025035699999999999 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.356 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173373+0000", 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.205496927 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:188 3.7s0 3:e5b49aa4:::smithi14230322-32:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558569+0000", 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.788924295000001, 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50916866999999999, 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.357 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 188 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558569+0000", 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.358 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558569+0000", 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558570+0000", 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2499999999999995e-07 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558621+0000", 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1082999999999999e-05 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.359 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558629+0000", 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9529999999999999e-06 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558634+0000", 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7360000000000001e-06 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558665+0000", 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1099000000000002e-05 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558669+0000", 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3610000000000003e-06 2024-04-18T23:11:37.360 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654665+0000", 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.095996355000000005 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654801+0000", 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013551599999999999 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.655971+0000", 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011700059999999999 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.361 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.657081+0000", 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001109737 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067667+0000", 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41058618899999999 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.067738+0000", 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1110000000000002e-05 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.362 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:189 3.bs0 3:d498df52:::smithi14230322-7:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.788810378000001, 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.91155819000000005, 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 189 2024-04-18T23:11:37.363 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558684+0000", 2024-04-18T23:11:37.364 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1799999999999995e-07 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558687+0000", 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1920000000000001e-06 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558694+0000", 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.452e-06 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658767+0000", 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100072129 2024-04-18T23:11:37.365 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757920+0000", 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099153626999999994 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757967+0000", 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6962999999999997e-05 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171274+0000", 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41330696700000003 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.366 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173905+0000", 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002630828 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470100+0000", 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29619521399999998 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470241+0000", 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014119999999999999 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.367 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:190 3.bs0 3:d498df52:::smithi14230322-7:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.658782+0000", 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.688711527999999, 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.51341810700000001, 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 190 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.368 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658782+0000", 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658782+0000", 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658783+0000", 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1209999999999999e-06 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.369 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658789+0000", 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4349999999999996e-06 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658798+0000", 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6099999999999995e-06 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658805+0000", 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4489999999999996e-06 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.370 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760577+0000", 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101772846 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760624+0000", 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6789000000000002e-05 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.172000+0000", 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41137571899999997 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.371 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.172095+0000", 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4592000000000005e-05 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.172200+0000", 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105546 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:197 3.7s0 3:e60e3c0c:::smithi14230322-6:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.758927+0000", 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.588567062999999, 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40981388400000002, 2024-04-18T23:11:37.372 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 197 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758927+0000", 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.373 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758927+0000", 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758928+0000", 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0470000000000001e-06 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758932+0000", 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8219999999999997e-06 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.374 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758939+0000", 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9280000000000004e-06 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758944+0000", 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4700000000000004e-06 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760155+0000", 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001211073 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760309+0000", 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015378399999999999 2024-04-18T23:11:37.375 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.760963+0000", 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00065410399999999997 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863295+0000", 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102332388 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168687+0000", 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30539211500000002 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.376 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.168740+0000", 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3152999999999997e-05 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:198 3.7s0 3:e60e3c0c:::smithi14230322-6:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.588535937000003, 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41079902800000001, 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.377 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 198 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.378 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758958+0000", 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9799999999999999e-07 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758960+0000", 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.623e-06 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758964+0000", 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4819999999999996e-06 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.379 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758967+0000", 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4570000000000001e-06 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761240+0000", 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002272081 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761347+0000", 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010780099999999999 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.380 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.761749+0000", 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00040184299999999998 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863298+0000", 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101549268 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169722+0000", 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.306423995 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.381 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169757+0000", 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4180000000000001e-05 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:212 3.7s0 3:e405d834:::smithi14230322-14:head [create,setxattr XNeZS[lOPtVca\\Pa[\\rdfbinb[bcjhtYSpmcafZm (1928) in=40b,setxattr TfflYYqPnps]lo^qSlp___TitNdd`]kaPTVOTcsh (1648) in=40b,setxattr TNb_onpY[j_dkhfmnWtm]Yktg]]RNtbQe[OUTNlX (770) in=40b,setxattr ZcgnkhY\\orTYaZh]PPldj`SNq`p\\cX`k]h\\bZY_Y (700) in=40b,setxattr gPQVnqhpsVhW_kq`QnafXrlRZtZ\\YnRftgQTishR (595) in=40b,setxattr U`XqYi[WhoqnpSPRc\\RhnppZp^qY\\qUfjpqs[r^a (1892) in=40b,setxattr cTmhd`pWgrRf_VfVgOtgeOP^ShP]lsOtpoTj[\\gl (418) in=40b,setxattr aa_taYTfslg`lRRrOkkraN\\XiW^V\\ljl]NqXin`a (314) in=40b,setxattr ibl]`igtgVThY^Qse\\okYYUtTmolX\\W]p]Q[P\\il (1403) in=40b,setxattr \\ZabSO[UUqeZa`hddNh[ZjaRRgUZoQtkn\\QdsX]W (1541) in=40b,setxattr jUkQUPTNbo`rZNmYlUmhb]ojc]aeheoVRfp]bPWf (5052) in=40b,setxattr Ofj`oQ^cR]da^SPsTaXmmPse\\OZ`VSqQ]U`efsZl (419) in=40b,setxattr a[cdkWos\\OrUVPciUaa^_^`lWVTScffiWkd\\[kgf (1261) in=40b,setxattr jXb]\\WhbatdZ[qTTiZO]kPd^`l[kajOYiQ[iqSYc (244) in=40b,setxattr \\im]jQhbanUn\\_^h`jaU]W`ReaZN_je[]kgg (1578) in=36b,setxattr ZbosoRORS^_de]rN^_daOiha\\Q\\eRae^[sQUjXsV (9120) in=40b,setxattr gcn]XpZUkOZr\\pkrVVZpYiR\\alh_PaZa_S[PYcn] (384) in=40b,setxattr \\^qUOaNXSTjtmafhfbo\\gseTTkV`mgnacYq^bkcc (34) in=40b,setxattr iUa^mOQ]TqjVSUU_nXfY]f`NjljiiaYbrm[o]aes (1611) in=40b,setxattr tXaQPQbc^aiV[^reVOPpib^caoWf^XgqilO^_g[k (1016) in=40b] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 36.588520598000002, 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41097265900000002, 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.382 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 212 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.383 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.758973+0000", 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9099999999999999e-07 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759003+0000", 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9641999999999998e-05 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759008+0000", 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.271e-06 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.384 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.759012+0000", 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1359999999999999e-06 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.862010+0000", 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102997908 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.862080+0000", 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9305000000000001e-05 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.385 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.862724+0000", 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00064440199999999997 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.863536+0000", 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00081221099999999997 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169861+0000", 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30632486799999997 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.169946+0000", 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4524999999999995e-05 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:279 3.7s0 3:ea1bab24:::smithi14230322-36:head [copy-from ver 15] snapc 6=[6,5,4] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.579501+0000", 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.767992460999999, 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.81417189400000001, 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.387 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 279 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579501+0000", 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579501+0000", 2024-04-18T23:11:37.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579503+0000", 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.798e-06 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579508+0000", 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4219999999999999e-06 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579520+0000", 2024-04-18T23:11:37.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1833e-05 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579531+0000", 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1080999999999999e-05 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677050+0000", 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097518821000000006 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677182+0000", 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013219100000000001 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091592+0000", 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41440982100000001 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.095981+0000", 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043885729999999998 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.393572+0000", 2024-04-18T23:11:37.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29759129499999998 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.393673+0000", 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101059 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:280 3.7s0 3:ea1bab24:::smithi14230322-36:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.579552+0000", 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.767941867000001, 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.51371551100000001, 2024-04-18T23:11:37.392 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 280 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579552+0000", 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579552+0000", 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579552+0000", 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8500000000000001e-07 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579555+0000", 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6390000000000001e-06 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579564+0000", 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1130000000000007e-06 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579571+0000", 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5570000000000001e-06 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677388+0000", 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097817297999999997 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:37.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677445+0000", 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7309999999999998e-05 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.093070+0000", 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41562443500000001 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.093174+0000", 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104144 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.093267+0000", 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3430000000000005e-05 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:322 3.bs0 3:d447fef0:::smithi14230322-49:8 [read 0~819200,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:10.608558+0000", 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.738935575999999, 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.44893157099999997, 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.397 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 322 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608558+0000", 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608558+0000", 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.398 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608559+0000", 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3300000000000002e-07 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608562+0000", 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8029999999999998e-06 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608572+0000", 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8749999999999995e-06 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.399 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608580+0000", 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1089999999999998e-06 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608674+0000", 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4075999999999998e-05 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.608678+0000", 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5860000000000001e-06 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.400 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.913531+0000", 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30485333599999997 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:10.913701+0000", 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016934599999999999 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.057490+0000", 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.14378880699999999 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.401 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:324 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~0] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:11.423978+0000", 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 25.923516033999999, 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41043667700000003, 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 324 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.402 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.423978+0000", 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.423978+0000", 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.423980+0000", 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9000000000000002e-06 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.403 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.423992+0000", 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1823e-05 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424039+0000", 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6909000000000001e-05 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424056+0000", 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6762999999999999e-05 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.404 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424196+0000", 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000140063 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424379+0000", 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018270600000000001 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.426794+0000", 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002415122 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.405 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.432886+0000", 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060920269999999999 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.834286+0000", 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40140048 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.834414+0000", 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012788400000000001 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.406 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:325 3.bs0 3:d447fef0:::smithi14230322-49:head [rollback 7] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:11.424087+0000", 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 25.923406863, 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41134601399999998, 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 325 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.407 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424087+0000", 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424087+0000", 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424088+0000", 2024-04-18T23:11:37.408 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3230000000000001e-06 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424100+0000", 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1559999999999999e-05 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424119+0000", 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9341999999999999e-05 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424134+0000", 2024-04-18T23:11:37.409 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5284000000000001e-05 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.428226+0000", 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0040921229999999996 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.428477+0000", 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025047399999999999 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.410 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.429867+0000", 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013900550000000001 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.432891+0000", 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0030243679999999999 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.835231+0000", 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40233970299999999 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.835433+0000", 2024-04-18T23:11:37.411 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00020178199999999999 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:326 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~0] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:11.424155+0000", 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 25.923338358999999, 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.51120384900000004, 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.412 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 326 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424155+0000", 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424155+0000", 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.413 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424156+0000", 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.237e-06 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424162+0000", 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.057e-06 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424179+0000", 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7405999999999999e-05 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.414 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.424192+0000", 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3553000000000001e-05 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.430657+0000", 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0064646080000000002 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.430911+0000", 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025408400000000002 2024-04-18T23:11:37.415 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.432114+0000", 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012027559999999999 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.433102+0000", 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00098769500000000002 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935230+0000", 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50212803900000003 2024-04-18T23:11:37.416 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935359+0000", 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000129414 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:334 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~704512] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.636133+0000", 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 24.711360526, 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50100039200000002, 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.417 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 334 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.636133+0000", 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.418 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.636133+0000", 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.636136+0000", 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6410000000000002e-06 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.637067+0000", 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093128099999999995 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.419 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.637100+0000", 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3339e-05 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.637118+0000", 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7377999999999999e-05 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.829485+0000", 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19236683600000001 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.829658+0000", 2024-04-18T23:11:37.420 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017355300000000001 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.831259+0000", 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0016010309999999999 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.934664+0000", 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103405119 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.137039+0000", 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20237438399999999 2024-04-18T23:11:37.421 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.137133+0000", 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4829999999999998e-05 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:335 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~376832] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.638777+0000", 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 24.708716865, 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.49869328600000001, 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.422 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 335 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.638777+0000", 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.423 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.638777+0000", 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.638779+0000", 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3309999999999998e-06 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639167+0000", 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038792399999999999 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639190+0000", 2024-04-18T23:11:37.424 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3383e-05 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639204+0000", 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3715e-05 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.932912+0000", 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29370825099999998 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933100+0000", 2024-04-18T23:11:37.425 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018750599999999999 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933966+0000", 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00086649399999999999 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.136557+0000", 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20259101099999999 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.426 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.137391+0000", 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083378900000000004 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.137470+0000", 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8881999999999995e-05 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:336 3.bs0 3:d447fef0:::smithi14230322-49:head [setxattr _header (56) in=7b] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.639228+0000", 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 24.708265768, 2024-04-18T23:11:37.427 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.59823371800000003, 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 336 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639228+0000", 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.428 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639228+0000", 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639229+0000", 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6599999999999994e-07 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639234+0000", 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4240000000000001e-06 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.429 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639246+0000", 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2208000000000001e-05 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.639254+0000", 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5469999999999999e-06 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.033282+0000", 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39402777999999999 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.033456+0000", 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017395900000000001 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.034378+0000", 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00092277099999999996 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.136559+0000", 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102180697 2024-04-18T23:11:37.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.237346+0000", 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100786389 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.237462+0000", 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115977 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.432 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:475 3.7s0 3:e5b49aa4:::smithi14230322-32:head [append 0~475136] snapc 12=[12,11,10,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:27.575024+0000", 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.7724692229999999, 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50953762700000005, 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 475 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.575024+0000", 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.575024+0000", 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.575026+0000", 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.007e-06 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.575595+0000", 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00056879800000000002 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.575632+0000", 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6773000000000002e-05 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.575652+0000", 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0023000000000001e-05 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.582594+0000", 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0069422520000000003 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.582714+0000", 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120272 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.583628+0000", 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00091340499999999995 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.436 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.660038+0000", 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.076409925000000004 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:28.084382+0000", 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.42434453900000002 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:28.084562+0000", 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017963300000000001 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:476 3.7s0 3:e5b49aa4:::smithi14230322-32:head [append 0~294912] snapc 12=[12,11,10,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e62)", 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:27.578673+0000", 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.7688204849999991, 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.506244166, 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 476 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.578673+0000", 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.578673+0000", 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.578676+0000", 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6539999999999999e-06 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.579057+0000", 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038134099999999999 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.579083+0000", 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5868999999999999e-05 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.679282+0000", 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10019858399999999 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.679391+0000", 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010929099999999999 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.679612+0000", 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000220931 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.680813+0000", 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012016970000000001 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.441 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.784481+0000", 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103667918 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:28.084746+0000", 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30026433899999999 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:28.084917+0000", 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017154200000000001 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:478 3.7s0 3:e5b49aa4:::smithi14230322-32:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e62)", 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:27.679429+0000", 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.6680647949999994, 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.60875793, 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 478 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.679429+0000", 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.679429+0000", 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.679430+0000", 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4100000000000001e-06 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.679437+0000", 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8970000000000002e-06 2024-04-18T23:11:37.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.679459+0000", 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1498999999999999e-05 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.679474+0000", 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5333e-05 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.685227+0000", 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057535049999999999 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.685414+0000", 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00018623199999999999 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:28.086218+0000", 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40080421100000002 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:28.086476+0000", 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025784000000000001 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.446 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:28.288187+0000", 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201711003 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:508 3.7s0 3:e97b01fb:::smithi14230322-37:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:31.546993+0000", 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.8005003090000002, 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.60079335300000003, 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.447 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 508 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.546993+0000", 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.546993+0000", 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.546995+0000", 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3230000000000001e-06 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.547001+0000", 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5239999999999997e-06 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.547020+0000", 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9162999999999999e-05 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.547036+0000", 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6072000000000001e-05 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.631314+0000", 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.084278049999999993 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.631324+0000", 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2189999999999993e-06 2024-04-18T23:11:37.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.750919+0000", 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.119595325 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.751075+0000", 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015589600000000001 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.947018+0000", 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19594341300000001 2024-04-18T23:11:37.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.947177+0000", 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015912599999999999 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:32.147787+0000", 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20060924199999999 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.452 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:37.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:11:37.455 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:37.451+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:37.554 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:37.550+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:37.558 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.0 to 1 (10000) 2024-04-18T23:11:37.560 INFO:tasks.rados.rados.0.smithi142.stdout:230: done (5 left) 2024-04-18T23:11:37.561 INFO:tasks.rados.rados.0.smithi142.stdout:231: snap_remove snap 7 2024-04-18T23:11:37.582 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-18T23:11:37.629 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:37.629 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 70/68 ECSubWriteReply(tid=84, last_complete=70'42, committed=1, applied=1))", 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:37.462972+0000", 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.16726332199999999, 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16728565000000001, 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.630 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.462972+0000", 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.462972+0000", 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.462974+0000", 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6899999999999999e-06 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.631 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.462983+0000", 2024-04-18T23:11:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5440000000000004e-06 2024-04-18T23:11:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.463009+0000", 2024-04-18T23:11:37.632 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5729e-05 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.563085+0000", 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10007582399999999 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.633 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDPGPushReply(3.5s1 70/68 [PushReplyOp(3:a09a6894:::smithi14230322-25:11)])", 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:37.563253+0000", 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.066982733000000003, 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.067038003999999998, 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.563253+0000", 2024-04-18T23:11:37.634 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.563253+0000", 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.563254+0000", 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.082e-06 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.563261+0000", 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2840000000000001e-06 2024-04-18T23:11:37.635 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.563276+0000", 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4844999999999999e-05 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.563285+0000", 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5130000000000002e-06 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.636 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs1 71/31 ECSubWrite(tid=57, reqid=osd.11.0:57, at_version=71'27, trim_to=0'0, roll_forward_to=69'23))", 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:37.583985+0000", 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.046250476999999998, 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.046334230999999997, 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.583985+0000", 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.637 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.583985+0000", 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.583986+0000", 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8599999999999997e-07 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.584014+0000", 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8285000000000001e-05 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.638 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.584028+0000", 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3712e-05 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.584037+0000", 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6639999999999993e-06 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 3 2024-04-18T23:11:37.639 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:37.645 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:11:37.655 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:37.658+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:37.756 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:37.754+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:37.828 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:37.828 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:37.828 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:37.828 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:37.828 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:37.843 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:11:37.857 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:37.855+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:37.958 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:37.959+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:37.964 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:37.964 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:37.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:37.964 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:37.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.964 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:194 3.4s0 3:2a56ff57:::smithi14230322-24:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:37.964 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.402797151999998, 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41191497500000002, 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 194 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558781+0000", 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8059999999999999e-06 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558789+0000", 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1340000000000004e-06 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558806+0000", 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6300999999999999e-05 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558814+0000", 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0099999999999995e-06 2024-04-18T23:11:37.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558882+0000", 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8281999999999994e-05 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558889+0000", 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2359999999999998e-06 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661030+0000", 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102141155 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661144+0000", 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113853 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766467+0000", 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.105323126 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.768352+0000", 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018843359999999999 2024-04-18T23:11:37.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.970633+0000", 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.202281391 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.970694+0000", 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1345000000000003e-05 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.fs4 51/31 ECSubWrite(tid=23, reqid=client.4568.0:384, at_version=51'4, trim_to=0'0, roll_forward_to=35'2))", 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:16.771818+0000", 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 21.189758442999999, 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30501938899999997, 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.771818+0000", 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.771818+0000", 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.771819+0000", 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7700000000000004e-07 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.772071+0000", 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000252117 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.772122+0000", 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0988e-05 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.872277+0000", 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100154665 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.972546+0000", 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100269264 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:16.972562+0000", 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6651999999999999e-05 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:17.076772+0000", 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104209521 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:17.076837+0000", 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5505000000000004e-05 2024-04-18T23:11:37.974 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:401 3.5s1 3:a6fe874e:::smithi14230322-33:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e53)", 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:18.504265+0000", 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.457311249, 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30516069299999998, 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.975 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 401 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504265+0000", 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504265+0000", 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504266+0000", 2024-04-18T23:11:37.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6799999999999999e-07 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504272+0000", 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1249999999999998e-06 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504282+0000", 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3640000000000005e-06 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504290+0000", 2024-04-18T23:11:37.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6309999999999992e-06 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504377+0000", 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6607999999999995e-05 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.504382+0000", 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6319999999999996e-06 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.608095+0000", 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103712123 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.608151+0000", 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6712000000000002e-05 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.809079+0000", 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20092789899999999 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.809191+0000", 2024-04-18T23:11:37.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000112013 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:18.809426+0000", 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000234818 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:469 3.5s1 3:a09a6894:::smithi14230322-25:head [copy-from ver 15] snapc 11=[11,10,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.635166+0000", 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.326410274000001, 2024-04-18T23:11:37.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82619366100000002, 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 469 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635166+0000", 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635166+0000", 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635169+0000", 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0400000000000001e-06 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635184+0000", 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4926000000000001e-05 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635212+0000", 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7597e-05 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635225+0000", 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3643e-05 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635366+0000", 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014097400000000001 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635381+0000", 2024-04-18T23:11:37.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4331000000000001e-05 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.638693+0000", 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033120110000000001 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.638809+0000", 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115821 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957052+0000", 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.31824379899999999 2024-04-18T23:11:37.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.961730+0000", 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0046779459999999997 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.460885+0000", 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.49915458400000001 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.461360+0000", 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00047498899999999999 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.985 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:470 3.5s1 3:a09a6894:::smithi14230322-25:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.635258+0000", 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.326318755000001, 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.32473432699999999, 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 470 2024-04-18T23:11:37.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635258+0000", 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635258+0000", 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635259+0000", 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8599999999999997e-07 2024-04-18T23:11:37.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635263+0000", 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0029999999999996e-06 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635274+0000", 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1423999999999999e-05 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635299+0000", 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5378999999999999e-05 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635499+0000", 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00019945799999999999 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635508+0000", 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0769999999999997e-06 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.639123+0000", 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003614992 2024-04-18T23:11:37.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.639176+0000", 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3245000000000003e-05 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958957+0000", 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.31978082200000002 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.959259+0000", 2024-04-18T23:11:37.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030184899999999998 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.959992+0000", 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00073319200000000002 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:471 3.5s1 3:a832c25e:::smithi14230322-46:head [copy-from ver 21] snapc 11=[11,10,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.635334+0000", 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.326242263999999, 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.52312541499999998, 2024-04-18T23:11:37.991 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 471 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635334+0000", 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635334+0000", 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635335+0000", 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5199999999999996e-07 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635340+0000", 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4929999999999998e-06 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635351+0000", 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0845e-05 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635358+0000", 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1559999999999998e-06 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635569+0000", 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021029600000000001 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635581+0000", 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2456000000000001e-05 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.639267+0000", 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0036858080000000001 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.639421+0000", 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015406999999999999 2024-04-18T23:11:37.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.844361+0000", 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20493995800000001 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.846386+0000", 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020250310000000001 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.158122+0000", 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.31173636799999999 2024-04-18T23:11:37.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.158460+0000", 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000337282 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs1 61/31 ECSubWrite(tid=42, reqid=osd.11.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=61'14))", 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.958302+0000", 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.003274806, 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38844035799999999, 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:37.997 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958302+0000", 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958302+0000", 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958306+0000", 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3170000000000003e-06 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958413+0000", 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000107431 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958531+0000", 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117197 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:37.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958575+0000", 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4651999999999999e-05 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.143952+0000", 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.18537637400000001 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.143979+0000", 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7772000000000001e-05 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.346690+0000", 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.202710693 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.346742+0000", 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1922000000000002e-05 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:548 3.5s1 3:a6fe874e:::smithi14230322-33:head [append 0~0] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T23:11:38.001 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.531316+0000", 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.4302605859999999, 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.42558847100000002, 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 548 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531316+0000", 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531316+0000", 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531318+0000", 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531328+0000", 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7599999999999997e-06 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531371+0000", 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3377999999999997e-05 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531383+0000", 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.223e-05 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547544+0000", 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016161225000000001 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547703+0000", 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015882900000000001 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.548679+0000", 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00097595699999999995 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.756002+0000", 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20732305100000001 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956635+0000", 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20063264 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956904+0000", 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000269497 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.006 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:549 3.5s1 3:a6fe874e:::smithi14230322-33:head [rollback 13] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.531403+0000", 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.430173199, 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.52622381100000004, 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 549 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.007 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531403+0000", 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531403+0000", 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531404+0000", 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8599999999999998e-07 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.008 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531409+0000", 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0780000000000003e-06 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531417+0000", 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5499999999999997e-06 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531424+0000", 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1570000000000003e-06 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.009 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.549557+0000", 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018133732999999999 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.549692+0000", 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013470600000000001 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.550566+0000", 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00087358599999999998 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.856154+0000", 2024-04-18T23:11:38.010 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30558865499999999 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057374+0000", 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20122003399999999 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057627+0000", 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025272599999999999 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.011 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:550 3.5s1 3:a6fe874e:::smithi14230322-33:head [append 0~0] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.531438+0000", 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.430138132, 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.92696160500000002, 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 550 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.012 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531438+0000", 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531438+0000", 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531439+0000", 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7100000000000002e-07 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.013 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531441+0000", 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531448+0000", 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.844e-06 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531453+0000", 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0459999999999996e-06 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.014 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.058137+0000", 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.52668325500000002 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.058247+0000", 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110487 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.058710+0000", 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046313300000000001 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.015 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.062805+0000", 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004094749 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.457788+0000", 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39498330100000001 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.458400+0000", 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00061167700000000003 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.016 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.5s1 69/68 ECSubRead(tid=76, to_read={3:a832c25e:::smithi14230322-46:11=0,262144,0}, subchunks={3:a832c25e:::smithi14230322-46:11=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.4141788049999999, 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.51415498900000001, 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.017 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.018 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.550808+0000", 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003410153 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.550820+0000", 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1797000000000001e-05 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.059003+0000", 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50818370400000001 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.019 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.061553+0000", 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025493349999999998 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=77, last_complete=69'40, committed=1, applied=1))", 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.551303+0000", 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.4102730700000001, 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30501677900000002, 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.020 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.551303+0000", 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.551303+0000", 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.021 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.551305+0000", 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1510000000000001e-06 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.551310+0000", 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6450000000000002e-06 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.551324+0000", 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4234e-05 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.022 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651482+0000", 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100157943 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.856297+0000", 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20481502800000001 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.856320+0000", 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2778000000000001e-05 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.023 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=77, last_complete=54'34, committed=1, applied=1))", 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.553039+0000", 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.4085377399999999, 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30311775800000001, 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553039+0000", 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.024 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553039+0000", 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553039+0000", 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.13e-07 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553041+0000", 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8959999999999999e-06 2024-04-18T23:11:38.025 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553050+0000", 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5199999999999997e-06 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553057+0000", 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.0310000000000001e-06 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.856129+0000", 2024-04-18T23:11:38.026 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.303072491 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.856157+0000", 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7307000000000001e-05 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=77, last_complete=69'40, committed=1, applied=1))", 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.553100+0000", 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.408476936, 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30315550200000002, 2024-04-18T23:11:38.027 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553100+0000", 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553100+0000", 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.028 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553100+0000", 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8000000000000001e-07 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553103+0000", 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.242e-06 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553112+0000", 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3170000000000002e-06 2024-04-18T23:11:38.029 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.553118+0000", 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8009999999999998e-06 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.856218+0000", 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30309988700000001 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.856255+0000", 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6875000000000002e-05 2024-04-18T23:11:38.030 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=78, last_complete=69'41, committed=1, applied=1))", 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.555925+0000", 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.405651376, 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40055818999999998, 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.031 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555925+0000", 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555925+0000", 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555926+0000", 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.015e-06 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.032 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555930+0000", 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6990000000000001e-06 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555945+0000", 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5007999999999999e-05 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555957+0000", 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.19e-05 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956440+0000", 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.400482909 2024-04-18T23:11:38.033 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956483+0000", 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3659000000000003e-05 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=77, last_complete=69'40, committed=1, applied=1))", 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.633103+0000", 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.328473974, 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.32381246899999999, 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.034 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633103+0000", 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633103+0000", 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633104+0000", 2024-04-18T23:11:38.035 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5319999999999999e-06 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633112+0000", 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4320000000000002e-06 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633133+0000", 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1554999999999999e-05 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633148+0000", 2024-04-18T23:11:38.036 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5045000000000001e-05 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956547+0000", 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.32339901100000001 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956915+0000", 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036789400000000001 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.037 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=78, last_complete=69'41, committed=1, applied=1))", 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.633355+0000", 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.328221267, 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.424701157, 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633355+0000", 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.038 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633355+0000", 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633356+0000", 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0999999999999999e-07 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633360+0000", 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7919999999999999e-06 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633370+0000", 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0088000000000001e-05 2024-04-18T23:11:38.039 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633378+0000", 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9899999999999997e-06 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057298+0000", 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.42392059799999998 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.058056+0000", 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00075817899999999999 2024-04-18T23:11:38.040 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=78, last_complete=69'41, committed=1, applied=1))", 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.651505+0000", 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.310071309, 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40577508200000001, 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651505+0000", 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.041 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651505+0000", 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651506+0000", 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7199999999999999e-07 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651509+0000", 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.641e-06 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.042 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651518+0000", 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2940000000000006e-06 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651523+0000", 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9019999999999998e-06 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057243+0000", 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40572015299999997 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.043 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057280+0000", 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7620000000000002e-05 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 70/68 ECSubWriteReply(tid=78, last_complete=54'34, committed=1, applied=1))", 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.654510+0000", 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.3070665969999999, 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30254083399999998, 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.044 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.654510+0000", 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.654510+0000", 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.654511+0000", 2024-04-18T23:11:38.045 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.654517+0000", 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0809999999999999e-06 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.654530+0000", 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2995e-05 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.046 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.654537+0000", 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4830000000000002e-06 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.957034+0000", 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30249654799999998 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.957051+0000", 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7007000000000002e-05 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.047 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 70/68 ECSubWriteReply(tid=78, last_complete=69'41, committed=1, applied=1))", 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.655398+0000", 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.3061785379999999, 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40178138000000002, 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655398+0000", 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.048 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655398+0000", 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655399+0000", 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1009999999999999e-06 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655406+0000", 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5300000000000002e-06 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.049 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655420+0000", 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4824000000000001e-05 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655436+0000", 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5600999999999999e-05 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057155+0000", 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40171905899999999 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.050 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.051 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.051 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057179+0000", 2024-04-18T23:11:38.051 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4264999999999999e-05 2024-04-18T23:11:38.051 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.051 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.051 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.051 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.051 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.051 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:38.052 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:11:38.100 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:38.098+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:38.161 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:38.155+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:38.180 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:38.180 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:11:38.180 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:38.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:11:38.261 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:38.258+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:38.359 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:38.360 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:38.362 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:38.363+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:38.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:11:38.464 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:38.463+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 33/31 ECSubWrite(tid=2, reqid=client.4568.0:19, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.967178275000002, 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102238363, 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.491 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523249+0000", 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8599999999999998e-07 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523292+0000", 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2746e-05 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.523307+0000", 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5223e-05 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623426+0000", 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10011853599999999 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623593+0000", 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016698900000000001 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623604+0000", 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1829e-05 2024-04-18T23:11:38.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624978+0000", 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0013738699999999999 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.625487+0000", 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00050868400000000002 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 34/31 ECSubWrite(tid=0, reqid=client.4568.0:37, at_version=34'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.623754+0000", 2024-04-18T23:11:38.495 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.866672979000001, 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102510163, 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623754+0000", 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623754+0000", 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623755+0000", 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5000000000000002e-07 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623927+0000", 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000172802 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623957+0000", 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9016000000000001e-05 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.623977+0000", 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0886e-05 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.724391+0000", 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100413961 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.724405+0000", 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4079e-05 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726222+0000", 2024-04-18T23:11:38.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001816151 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.726264+0000", 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2518000000000002e-05 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.866398865999997, 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10419479800000001, 2024-04-18T23:11:38.499 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.500 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624029+0000", 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4300000000000001e-07 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624060+0000", 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1025999999999997e-05 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624083+0000", 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3323e-05 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624104+0000", 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1089000000000001e-05 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725186+0000", 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101081146 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725195+0000", 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7829999999999993e-06 2024-04-18T23:11:38.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728164+0000", 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029689249999999999 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728223+0000", 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8563000000000002e-05 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 36/31 ECSubWrite(tid=67, reqid=client.4568.0:189, at_version=0'0, trim_to=0'0, roll_forward_to=36'17))", 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:01.470594+0000", 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 37.019833400000003, 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10183064999999999, 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470594+0000", 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470594+0000", 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470596+0000", 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0669999999999999e-06 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470621+0000", 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5222999999999999e-05 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470688+0000", 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6824999999999998e-05 2024-04-18T23:11:38.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470713+0000", 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5474999999999999e-05 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.570999+0000", 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10028558899999999 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.571026+0000", 2024-04-18T23:11:38.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7676000000000001e-05 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.572255+0000", 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012286549999999999 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.572424+0000", 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016914 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.507 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 42/31 ECSubWrite(tid=85, reqid=osd.4.0:85, at_version=42'27, trim_to=0'0, roll_forward_to=40'25))", 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.912694594000001, 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16837443799999999, 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577733+0000", 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6300000000000001e-07 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577758+0000", 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5099000000000001e-05 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577779+0000", 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0937999999999998e-05 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577792+0000", 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3227999999999999e-05 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744736+0000", 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.166943963 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744769+0000", 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2629000000000002e-05 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746075+0000", 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001306338 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746107+0000", 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.188e-05 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.6s4 43/31 ECSubWrite(tid=19, reqid=osd.0.0:18, at_version=0'0, trim_to=0'0, roll_forward_to=43'12))", 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:07.990532+0000", 2024-04-18T23:11:38.511 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 30.499894606000002, 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10125936100000001, 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990532+0000", 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990532+0000", 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990534+0000", 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4980000000000001e-06 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990553+0000", 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8848999999999999e-05 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990575+0000", 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2339000000000001e-05 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990587+0000", 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2340999999999999e-05 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990737+0000", 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015011700000000001 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.990774+0000", 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6745999999999997e-05 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:08.091745+0000", 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100970571 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:08.091792+0000", 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6900000000000002e-05 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 46/31 ECSubWrite(tid=113, reqid=client.4568.0:326, at_version=0'0, trim_to=0'0, roll_forward_to=46'31))", 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:11.935715+0000", 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 26.554712403, 2024-04-18T23:11:38.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102077656, 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935715+0000", 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935715+0000", 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935716+0000", 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.497e-06 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935731+0000", 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4958e-05 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935752+0000", 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0815999999999999e-05 2024-04-18T23:11:38.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:11.935765+0000", 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3481e-05 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.036081+0000", 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10031585699999999 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.036108+0000", 2024-04-18T23:11:38.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7282999999999999e-05 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.037695+0000", 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001586524 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.037792+0000", 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7239999999999997e-05 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 47/31 ECSubWrite(tid=26, reqid=client.4568.0:333, at_version=47'16, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 25.758542832, 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201860068, 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6299999999999999e-07 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732265+0000", 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038060399999999999 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732281+0000", 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6362e-05 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732292+0000", 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0817e-05 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.832471+0000", 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100179254 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.832486+0000", 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4911e-05 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933708+0000", 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101221567 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933744+0000", 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6390000000000002e-05 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 47/31 ECSubWrite(tid=27, reqid=client.4568.0:344, at_version=47'18, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 25.755895498000001, 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19924303900000001, 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734532+0000", 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734555+0000", 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3258999999999999e-05 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734568+0000", 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2568999999999999e-05 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734575+0000", 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6279999999999999e-06 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.833597+0000", 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099021993000000003 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.833605+0000", 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0620000000000001e-06 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933755+0000", 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10015038 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933774+0000", 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9474999999999999e-05 2024-04-18T23:11:38.527 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 47/31 ECSubWrite(tid=119, reqid=client.4568.0:336, at_version=0'0, trim_to=0'0, roll_forward_to=47'36))", 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:13.269515+0000", 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 25.220912336000001, 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10301724499999999, 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269515+0000", 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269515+0000", 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269517+0000", 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6529999999999998e-06 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269550+0000", 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2605999999999999e-05 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.269587+0000", 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7452e-05 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.369688+0000", 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10010050400000001 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.369783+0000", 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5655000000000005e-05 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.369795+0000", 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1705999999999999e-05 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.372486+0000", 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026909960000000002 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:13.372532+0000", 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5673000000000001e-05 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 57/31 ECSubWrite(tid=168, reqid=osd.4.0:168, at_version=57'60, trim_to=0'0, roll_forward_to=55'58))", 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.629337+0000", 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.861089721000001, 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.1034201, 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.532 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629337+0000", 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629337+0000", 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629337+0000", 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9799999999999999e-07 2024-04-18T23:11:38.533 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629364+0000", 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6950000000000001e-05 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629394+0000", 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9232000000000001e-05 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.629406+0000", 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1987000000000001e-05 2024-04-18T23:11:38.534 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.730744+0000", 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101338254 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.730777+0000", 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3071999999999998e-05 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.732693+0000", 2024-04-18T23:11:38.535 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019157060000000001 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.732757+0000", 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4601e-05 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 57/31 ECSubWrite(tid=169, reqid=osd.4.0:168, at_version=0'0, trim_to=0'0, roll_forward_to=57'60))", 2024-04-18T23:11:38.536 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:22.733631+0000", 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.756795925, 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101563061, 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733631+0000", 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733631+0000", 2024-04-18T23:11:38.537 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733632+0000", 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4479999999999999e-06 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733648+0000", 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5964000000000002e-05 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.538 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733672+0000", 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3731e-05 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733689+0000", 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7326999999999999e-05 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733840+0000", 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015033700000000001 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.539 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.733866+0000", 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6163e-05 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.835077+0000", 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101210974 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.835194+0000", 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117117 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.540 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 60/31 ECSubWrite(tid=38, reqid=client.4568.0:452, at_version=0'0, trim_to=0'0, roll_forward_to=60'10))", 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.352985+0000", 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 12.137442352000001, 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30133912600000001, 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.352985+0000", 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.541 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.352985+0000", 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.352986+0000", 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0109999999999999e-06 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.353002+0000", 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6235999999999998e-05 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.542 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.353020+0000", 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8451999999999999e-05 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.453102+0000", 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100081295 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.553350+0000", 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100248296 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.543 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.553384+0000", 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3729000000000002e-05 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654230+0000", 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100846175 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654324+0000", 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3931999999999994e-05 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.544 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 61/31 ECSubWrite(tid=183, reqid=osd.4.0:183, at_version=61'66, trim_to=0'0, roll_forward_to=58'62))", 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.746721+0000", 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.743705718999999, 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104157869, 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746721+0000", 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.545 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746721+0000", 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746723+0000", 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.302e-06 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746748+0000", 2024-04-18T23:11:38.546 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5795999999999998e-05 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746766+0000", 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7342000000000001e-05 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.846876+0000", 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10010988 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847103+0000", 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000227848 2024-04-18T23:11:38.547 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847132+0000", 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8132000000000001e-05 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.850847+0000", 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037155199999999999 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.850879+0000", 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2048999999999998e-05 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.548 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 61/31 ECSubWrite(tid=43, reqid=osd.2.0:43, at_version=61'31, trim_to=0'0, roll_forward_to=60'27))", 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.847226+0000", 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.643200454, 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201633386, 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847226+0000", 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.549 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847226+0000", 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847227+0000", 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847269+0000", 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2058999999999998e-05 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.550 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847292+0000", 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2299000000000001e-05 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.947376+0000", 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008486599999999 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.047557+0000", 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10018052099999999 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.551 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.047574+0000", 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6690000000000001e-05 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.048752+0000", 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011779469999999999 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.048860+0000", 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108194 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.552 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 61/31 ECSubWrite(tid=42, reqid=osd.11.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=61'14))", 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.957708+0000", 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.532718987999999, 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102450399, 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957708+0000", 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.553 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957708+0000", 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957710+0000", 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0729999999999999e-06 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957740+0000", 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9785999999999999e-05 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.554 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957788+0000", 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7787999999999997e-05 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.057883+0000", 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100095802 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.555 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.058041+0000", 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000157353 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.058072+0000", 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1646000000000002e-05 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.060085+0000", 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020130999999999999 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.556 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.060158+0000", 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2850999999999998e-05 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 61/31 ECSubWrite(tid=45, reqid=osd.2.0:45, at_version=61'33, trim_to=0'0, roll_forward_to=61'31))", 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:27.245450+0000", 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.244976471999999, 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10286067, 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.557 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.245450+0000", 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.245450+0000", 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.558 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.245452+0000", 2024-04-18T23:11:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.55e-06 2024-04-18T23:11:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.245510+0000", 2024-04-18T23:11:38.559 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8195000000000001e-05 2024-04-18T23:11:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.245546+0000", 2024-04-18T23:11:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5955999999999999e-05 2024-04-18T23:11:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.561 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.345690+0000", 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100143732 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.345837+0000", 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014752600000000001 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.345863+0000", 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5868999999999999e-05 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.562 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.348231+0000", 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002367924 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.348311+0000", 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9918000000000003e-05 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 66/31 ECSubWrite(tid=50, reqid=client.4568.0:497, at_version=0'0, trim_to=0'0, roll_forward_to=66'37))", 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:31.739659+0000", 2024-04-18T23:11:38.563 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 6.7507680900000002, 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101769599, 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.739659+0000", 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.739659+0000", 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.564 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.739661+0000", 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1339999999999998e-06 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.739689+0000", 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8399e-05 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.739729+0000", 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9752000000000002e-05 2024-04-18T23:11:38.565 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.739755+0000", 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5536000000000001e-05 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.839992+0000", 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100237191 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.840026+0000", 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3909999999999999e-05 2024-04-18T23:11:38.566 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.841331+0000", 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001305357 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.841428+0000", 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7319999999999997e-05 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.567 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 70/31 ECSubWrite(tid=58, reqid=client.4568.0:552, at_version=70'44, trim_to=0'0, roll_forward_to=69'42))", 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.671048+0000", 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.819378591, 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101597074, 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.671048+0000", 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.671048+0000", 2024-04-18T23:11:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.671050+0000", 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6080000000000001e-06 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.671266+0000", 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000216264 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.671282+0000", 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5515999999999998e-05 2024-04-18T23:11:38.569 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.671294+0000", 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2300000000000001e-05 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.771465+0000", 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10017100900000001 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.771474+0000", 2024-04-18T23:11:38.570 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9530000000000005e-06 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.772607+0000", 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011334240000000001 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.772645+0000", 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8000000000000002e-05 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.571 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 71/31 ECSubWrite(tid=212, reqid=osd.4.0:212, at_version=71'72, trim_to=0'0, roll_forward_to=69'70))", 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:37.669823+0000", 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.82060385000000002, 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103943442, 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.669823+0000", 2024-04-18T23:11:38.572 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.669823+0000", 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.669823+0000", 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1500000000000001e-07 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.669844+0000", 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0805000000000002e-05 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.573 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.669856+0000", 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1721e-05 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.669865+0000", 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9940000000000006e-06 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.770022+0000", 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10015739799999999 2024-04-18T23:11:38.574 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.770036+0000", 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3788000000000001e-05 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.773731+0000", 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0036948699999999998 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.773767+0000", 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5651000000000001e-05 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.575 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:38.576 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:38.576 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:38.576 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T23:11:38.578 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:38.574+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:38.579 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:11:38.580 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.1 to 1 (10000) 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:231: done (5 left) 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 20 current snap is 21 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:224: finishing rollback tid 1 to smithi14230322-42 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:224: finishing rollback tid 2 to smithi14230322-42 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 42 v 54 (ObjNum 41 snap 0 seq_num 41) dirty exists 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:226: finishing copy_from racing read to smithi14230322-40 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:228: finishing write tid 1 to smithi14230322-13 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:232: seq_num 95 ranges {704512=442368,1146880=540672,1687552=622592} 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:232: writing smithi14230322-20 from 704512 to 1146880 tid 1 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:232: writing smithi14230322-20 from 1146880 to 1687552 tid 2 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:232: writing smithi14230322-20 from 1687552 to 2310144 tid 3 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:224: done (5 left) 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:233: copy_from oid 11 from oid 19 current snap is 21 2024-04-18T23:11:38.581 INFO:tasks.rados.rados.0.smithi142.stdout:228: finishing write tid 2 to smithi14230322-13 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:227: finishing rollback tid 0 to smithi14230322-33 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:228: finishing write tid 3 to smithi14230322-13 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:226: finishing copy_from to smithi14230322-40 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 40 v 4 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:227: finishing rollback tid 1 to smithi14230322-33 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:228: finishing write tid 4 to smithi14230322-13 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:228: finishing write tid 5 to smithi14230322-13 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:228: oid 13 updating version 0 to 46 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:228: oid 13 version 46 is already newer than 44 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:228: oid 13 version 46 is already newer than 43 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:228: oid 13 version 46 is already newer than 45 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 13 v 46 (ObjNum 94 snap 21 seq_num 94) dirty exists 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:228: left oid 13 (ObjNum 94 snap 21 seq_num 94) 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:227: finishing rollback tid 2 to smithi14230322-33 2024-04-18T23:11:38.582 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 33 v 42 (ObjNum 57 snap 4 seq_num 57) dirty exists 2024-04-18T23:11:38.583 INFO:tasks.rados.rados.0.smithi142.stdout:226: done (5 left) 2024-04-18T23:11:38.583 INFO:tasks.rados.rados.0.smithi142.stdout:227: done (4 left) 2024-04-18T23:11:38.583 INFO:tasks.rados.rados.0.smithi142.stdout:228: done (3 left) 2024-04-18T23:11:38.583 INFO:tasks.rados.rados.0.smithi142.stdout:234: delete oid 24 current snap is 21 2024-04-18T23:11:38.583 INFO:tasks.rados.rados.0.smithi142.stdout:229: finishing rollback tid 0 to smithi14230322-28 2024-04-18T23:11:38.583 INFO:tasks.rados.rados.0.smithi142.stdout:229: finishing rollback tid 1 to smithi14230322-28 2024-04-18T23:11:38.583 INFO:tasks.rados.rados.0.smithi142.stdout:229: finishing rollback tid 2 to smithi14230322-28 2024-04-18T23:11:38.583 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 28 v 24 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-18T23:11:38.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-18T23:11:38.678 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:38.675+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:38.711 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:38.711 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:38.711 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:38.716 INFO:tasks.rados.rados.0.smithi142.stdout:233: finishing copy_from to smithi14230322-11 2024-04-18T23:11:38.716 INFO:tasks.rados.rados.0.smithi142.stdout:233: got expected ENOENT (src dne) 2024-04-18T23:11:38.716 INFO:tasks.rados.rados.0.smithi142.stdout:233: finishing copy_from racing read to smithi14230322-11 2024-04-18T23:11:38.727 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:11:38.815 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:38.815+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:38.841 INFO:tasks.rados.rados.0.smithi142.stdout:232: finishing write tid 1 to smithi14230322-20 2024-04-18T23:11:38.871 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:38.879 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:38.878+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:38.897 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:38.898 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:38.898 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:38.898 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:38.898 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:38.912 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:11:38.912 INFO:tasks.rados.rados.0.smithi142.stdout:229: done (3 left) 2024-04-18T23:11:38.912 INFO:tasks.rados.rados.0.smithi142.stdout:233: done (2 left) 2024-04-18T23:11:38.912 INFO:tasks.rados.rados.0.smithi142.stdout:234: done (1 left) 2024-04-18T23:11:38.912 INFO:tasks.rados.rados.0.smithi142.stdout:235: delete oid 38 current snap is 21 2024-04-18T23:11:38.998 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:38.999+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:39.077 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 33/31 ECSubWrite(tid=2, reqid=client.4568.0:19, at_version=33'1, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.54927876, 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19915530100000001, 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.078 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528003+0000", 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528004+0000", 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.079 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528243+0000", 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000239244 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528271+0000", 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7535e-05 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528293+0000", 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2395000000000002e-05 2024-04-18T23:11:39.080 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.626035+0000", 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.097742512000000004 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.626059+0000", 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3739000000000001e-05 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727090+0000", 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101030651 2024-04-18T23:11:39.081 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.727158+0000", 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8415000000000001e-05 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.539394195, 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103172215, 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.082 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537888+0000", 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.083 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537889+0000", 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4839999999999999e-06 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537921+0000", 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2360999999999997e-05 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537948+0000", 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6421e-05 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.084 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.537963+0000", 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5218999999999999e-05 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638148+0000", 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100185015 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.638163+0000", 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4793e-05 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.085 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.640970+0000", 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028075439999999999 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.641060+0000", 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9377999999999997e-05 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:5, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:39.086 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.449192985000003, 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103548802, 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628089+0000", 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.087 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628091+0000", 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1129999999999999e-06 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628179+0000", 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7818000000000005e-05 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628219+0000", 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.0373000000000001e-05 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.088 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628243+0000", 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3549000000000001e-05 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628766+0000", 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052308499999999996 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628791+0000", 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4984999999999999e-05 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.089 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731546+0000", 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102755535 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731638+0000", 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1343999999999998e-05 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=2, reqid=client.4568.0:22, at_version=34'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:39.090 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.448980829, 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103381863, 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628301+0000", 2024-04-18T23:11:39.091 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628302+0000", 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628476+0000", 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017366199999999999 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628499+0000", 2024-04-18T23:11:39.092 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3025000000000001e-05 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628519+0000", 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0194999999999999e-05 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629371+0000", 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00085246300000000003 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.093 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629394+0000", 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3311000000000001e-05 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731650+0000", 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10225558899999999 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731683+0000", 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2654999999999998e-05 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.094 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 34/31 ECSubWrite(tid=3, reqid=client.4568.0:23, at_version=34'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.448283639000003, 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103947704, 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.095 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.628998+0000", 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629005+0000", 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7089999999999996e-06 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629040+0000", 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5133999999999999e-05 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.096 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629064+0000", 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4422000000000001e-05 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.629079+0000", 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4949e-05 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731181+0000", 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102101339 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.097 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.731199+0000", 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8175000000000001e-05 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.732877+0000", 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0016778050000000001 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.732946+0000", 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9171000000000006e-05 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.098 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 47/31 ECSubWrite(tid=22, reqid=osd.5.0:22, at_version=47'16, trim_to=0'0, roll_forward_to=46'14))", 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.623975+0000", 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.453306723000001, 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102688324, 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.099 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.623975+0000", 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.623975+0000", 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.623976+0000", 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9299999999999997e-07 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.100 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.624030+0000", 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4165000000000001e-05 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.624064+0000", 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4610999999999998e-05 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.724208+0000", 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10014355799999999 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.101 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.724385+0000", 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000177123 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.724414+0000", 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8600999999999999e-05 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.726569+0000", 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002155751 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.102 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.726663+0000", 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3821999999999997e-05 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 47/31 ECSubWrite(tid=116, reqid=client.4568.0:335, at_version=47'35, trim_to=0'0, roll_forward_to=47'33))", 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.935986+0000", 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 26.141295237000001, 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102410364, 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.103 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.935986+0000", 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.935986+0000", 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.935988+0000", 2024-04-18T23:11:39.104 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8729999999999999e-06 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936143+0000", 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015484299999999999 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.936174+0000", 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0571999999999998e-05 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036325+0000", 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10015107400000001 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.105 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036499+0000", 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017410200000000001 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.036514+0000", 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4586000000000001e-05 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038304+0000", 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017909309999999999 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.106 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.038397+0000", 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2383e-05 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 61/31 ECSubWrite(tid=180, reqid=osd.4.0:180, at_version=61'64, trim_to=0'0, roll_forward_to=58'62))", 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.646084+0000", 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.431197226, 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102581479, 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.107 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646084+0000", 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646084+0000", 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646085+0000", 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9299999999999996e-07 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.108 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646133+0000", 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.7995999999999999e-05 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646158+0000", 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4264999999999999e-05 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646180+0000", 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2523000000000001e-05 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.109 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646347+0000", 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000166768 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.646375+0000", 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8056e-05 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.748576+0000", 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102201088 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.110 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.748666+0000", 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9889999999999995e-05 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 61/31 ECSubWrite(tid=42, reqid=osd.2.0:42, at_version=61'29, trim_to=0'0, roll_forward_to=60'27))", 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.658283+0000", 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.418998872, 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.18780378, 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.111 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658283+0000", 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658283+0000", 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658285+0000", 2024-04-18T23:11:39.112 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7060000000000001e-06 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658326+0000", 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1149000000000002e-05 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658355+0000", 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9009e-05 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.113 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.114 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.658371+0000", 2024-04-18T23:11:39.114 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.59e-05 2024-04-18T23:11:39.114 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.757618+0000", 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.099247506999999999 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.757642+0000", 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3929000000000001e-05 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.115 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.846008+0000", 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.088365851999999995 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.846087+0000", 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8727999999999996e-05 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.116 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 61/31 ECSubWrite(tid=43, reqid=osd.2.0:43, at_version=61'31, trim_to=0'0, roll_forward_to=60'27))", 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.659216+0000", 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.41806527, 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.18816092300000001, 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659216+0000", 2024-04-18T23:11:39.117 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659216+0000", 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659218+0000", 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6559999999999999e-06 2024-04-18T23:11:39.118 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659261+0000", 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2432000000000003e-05 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659290+0000", 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9689e-05 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.119 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.659306+0000", 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5480000000000001e-05 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.845119+0000", 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.18581310400000001 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.845150+0000", 2024-04-18T23:11:39.120 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1334999999999997e-05 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.847302+0000", 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002151802 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.847377+0000", 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5425000000000004e-05 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.121 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 61/31 ECSubWrite(tid=186, reqid=osd.4.0:186, at_version=61'68, trim_to=0'0, roll_forward_to=61'66))", 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.120622588, 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201499973, 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.122 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.123 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956661+0000", 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7370000000000001e-06 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956723+0000", 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2564999999999995e-05 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956762+0000", 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8711999999999998e-05 2024-04-18T23:11:39.124 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.056877+0000", 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10011492900000001 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.057020+0000", 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142662 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.057046+0000", 2024-04-18T23:11:39.125 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6486e-05 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.158121+0000", 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10107499 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.158159+0000", 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7892000000000002e-05 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.126 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 61/31 ECSubWrite(tid=45, reqid=osd.2.0:45, at_version=61'33, trim_to=0'0, roll_forward_to=61'31))", 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.731694775999999, 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20243294100000001, 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.127 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345589+0000", 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7540000000000001e-06 2024-04-18T23:11:39.128 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345643+0000", 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4707999999999999e-05 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345662+0000", 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8870000000000001e-05 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345675+0000", 2024-04-18T23:11:39.129 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2482e-05 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.545813+0000", 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20013862499999999 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.545829+0000", 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5201999999999999e-05 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.130 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.547933+0000", 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021040389999999998 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.548020+0000", 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7261000000000004e-05 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.131 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 66/31 ECSubWrite(tid=49, reqid=client.4568.0:497, at_version=66'37, trim_to=0'0, roll_forward_to=65'35))", 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:31.536062+0000", 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 7.5412196820000004, 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10242389, 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.536062+0000", 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.132 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.536062+0000", 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.536062+0000", 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.65e-07 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.133 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.536204+0000", 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014166600000000001 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.536227+0000", 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2634999999999999e-05 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.536240+0000", 2024-04-18T23:11:39.134 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3365e-05 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.536393+0000", 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015285900000000001 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.536410+0000", 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7142e-05 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.135 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.638446+0000", 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102036315 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:31.638486+0000", 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9643e-05 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.136 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 69/31 ECSubWrite(tid=210, reqid=client.4568.0:545, at_version=69'70, trim_to=0'0, roll_forward_to=61'68))", 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.329688+0000", 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.7475941019999999, 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10226089100000001, 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.329688+0000", 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.137 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.329688+0000", 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.329689+0000", 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8700000000000001e-06 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.138 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.329732+0000", 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.2669000000000001e-05 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.329763+0000", 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1349000000000001e-05 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.429858+0000", 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100094235 2024-04-18T23:11:39.139 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.430012+0000", 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000154253 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.430032+0000", 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0159000000000001e-05 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.140 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.431878+0000", 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018462439999999999 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.431948+0000", 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0111999999999996e-05 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.141 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 70/31 ECSubWrite(tid=57, reqid=client.4568.0:551, at_version=70'43, trim_to=0'0, roll_forward_to=69'42))", 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.565666+0000", 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.5116158909999999, 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202564679, 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565666+0000", 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.142 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565666+0000", 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565667+0000", 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2699999999999998e-07 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.143 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565871+0000", 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020396499999999999 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565899+0000", 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8572e-05 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.666000+0000", 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100101172 2024-04-18T23:11:39.144 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.766291+0000", 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100290465 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.766307+0000", 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5979999999999999e-05 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.145 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.768156+0000", 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018494799999999999 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.768230+0000", 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4218e-05 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.146 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 70/31 ECSubWrite(tid=58, reqid=client.4568.0:552, at_version=70'44, trim_to=0'0, roll_forward_to=69'42))", 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.672281+0000", 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 2.405001124, 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102706965, 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.672281+0000", 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.147 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.672281+0000", 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.672282+0000", 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6729999999999999e-06 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.672808+0000", 2024-04-18T23:11:39.148 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00052541199999999995 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.672838+0000", 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0516000000000001e-05 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.672854+0000", 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5503e-05 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.149 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.773132+0000", 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10027797099999999 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.773150+0000", 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.789e-05 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.774933+0000", 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0017834680000000001 2024-04-18T23:11:39.150 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.774988+0000", 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4531999999999999e-05 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 70/31 ECSubWrite(tid=61, reqid=client.4568.0:554, at_version=0'0, trim_to=0'0, roll_forward_to=70'46))", 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.082659+0000", 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.9946226650000001, 2024-04-18T23:11:39.151 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20138941299999999, 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082659+0000", 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.152 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082659+0000", 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082661+0000", 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.694e-06 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082682+0000", 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1231999999999999e-05 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.153 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082712+0000", 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0273e-05 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.182872+0000", 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100159647 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.183044+0000", 2024-04-18T23:11:39.154 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000172356 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.183071+0000", 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7097000000000002e-05 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.283975+0000", 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100903359 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.155 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.284048+0000", 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3755000000000001e-05 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.9s1 71/68 ECSubRead(tid=147, to_read={3:9e84ac80:::smithi14230322-40:16=0,262144,0}, subchunks={3:9e84ac80:::smithi14230322-40:16=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.680353+0000", 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.3969286830000001, 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20058264000000001, 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.156 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680353+0000", 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680353+0000", 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.157 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680355+0000", 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.716e-06 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680372+0000", 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7346999999999998e-05 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.158 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680405+0000", 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2685999999999998e-05 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.780512+0000", 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100106973 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.880742+0000", 2024-04-18T23:11:39.159 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100230139 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.880936+0000", 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019377899999999999 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.160 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 71/70 ECSubWrite(tid=33, reqid=client.4568.0:557, at_version=71'23, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.885116+0000", 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.1921654909999999, 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10297084500000001, 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.885116+0000", 2024-04-18T23:11:39.161 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.885116+0000", 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.885118+0000", 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.877e-06 2024-04-18T23:11:39.162 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.885174+0000", 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.6147999999999998e-05 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.885211+0000", 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6699999999999998e-05 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.163 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.985298+0000", 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100086762 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.985420+0000", 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000122683 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.985435+0000", 2024-04-18T23:11:39.164 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4219999999999999e-05 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.987983+0000", 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025480709999999998 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.988087+0000", 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104384 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.165 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 72/31 ECSubWrite(tid=66, reqid=osd.2.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:38.716258+0000", 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.361023292, 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103701475, 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.166 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716258+0000", 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716258+0000", 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716259+0000", 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5300000000000003e-07 2024-04-18T23:11:39.167 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716308+0000", 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.8542999999999999e-05 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716333+0000", 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5442000000000001e-05 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716351+0000", 2024-04-18T23:11:39.168 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8243000000000001e-05 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.817395+0000", 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10104314 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.817409+0000", 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4440999999999999e-05 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.169 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.819889+0000", 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024799850000000001 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.819960+0000", 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0827999999999993e-05 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.170 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:39.171 INFO:tasks.rados.rados.0.smithi142.stdout:232: finishing write tid 2 to smithi14230322-20 2024-04-18T23:11:39.172 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:11:39.173 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:39.170+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:39.222 INFO:tasks.rados.rados.0.smithi142.stdout:232: finishing write tid 3 to smithi14230322-20 2024-04-18T23:11:39.239 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:39.239 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:39.265 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:39.265 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:39.273 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:39.271+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:39.296 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:39.296 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:39.322 INFO:tasks.rados.rados.0.smithi142.stdout:232: finishing write tid 4 to smithi14230322-20 2024-04-18T23:11:39.329 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:39.329 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:39.346 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:39.346 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.cs6 72/31 ECSubWrite(tid=67, reqid=client.4568.0:564, at_version=0'0, trim_to=0'0, roll_forward_to=72'35))", 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:39.325372+0000", 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.023407275000000002, 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.023445664000000001, 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.325372+0000", 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.325372+0000", 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.325373+0000", 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.556e-06 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.325394+0000", 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0999999999999999e-05 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.325421+0000", 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6316e-05 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.325436+0000", 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5065999999999999e-05 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 1 2024-04-18T23:11:39.349 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:39.361 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:11:39.407 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:39.407+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:39.421 INFO:tasks.rados.rados.0.smithi142.stdout:235: done (1 left) 2024-04-18T23:11:39.421 INFO:tasks.rados.rados.0.smithi142.stdout:236: setattr oid 48 current snap is 21 2024-04-18T23:11:39.425 INFO:tasks.rados.rados.0.smithi142.stdout:237: snap_remove snap 16 2024-04-18T23:11:39.453 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:39.453 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:39.474 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:39.474+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:39.481 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:39.481 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:39.481 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:39.481 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:39.481 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:39.488 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:39.496 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:11:39.508 INFO:tasks.rados.rados.0.smithi142.stdout:237: done (2 left) 2024-04-18T23:11:39.508 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 11 current snap is 21 2024-04-18T23:11:39.508 INFO:tasks.rados.rados.0.smithi142.stdout:238: seq_num 98 ranges {180224=491520,671744=589824,1261568=557056,1818624=16384} 2024-04-18T23:11:39.510 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.2 to 1 (10000) 2024-04-18T23:11:39.510 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-04-18T23:11:39.520 INFO:tasks.rados.rados.0.smithi142.stdout:238: writing smithi14230322-11 from 180224 to 671744 tid 1 2024-04-18T23:11:39.526 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:39.526 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:39.531 INFO:tasks.rados.rados.0.smithi142.stdout:238: writing smithi14230322-11 from 671744 to 1261568 tid 2 2024-04-18T23:11:39.540 INFO:tasks.rados.rados.0.smithi142.stdout:238: writing smithi14230322-11 from 1261568 to 1818624 tid 3 2024-04-18T23:11:39.541 INFO:tasks.rados.rados.0.smithi142.stdout:238: writing smithi14230322-11 from 1818624 to 1835008 tid 4 2024-04-18T23:11:39.541 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 50 current snap is 21 2024-04-18T23:11:39.541 INFO:tasks.rados.rados.0.smithi142.stdout:239: seq_num 99 ranges {32768=507904,540672=606208,1146880=557056} 2024-04-18T23:11:39.544 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-18T23:11:39.550 INFO:tasks.rados.rados.0.smithi142.stdout:239: writing smithi14230322-50 from 32768 to 540672 tid 1 2024-04-18T23:11:39.559 INFO:tasks.rados.rados.0.smithi142.stdout:239: writing smithi14230322-50 from 540672 to 1146880 tid 2 2024-04-18T23:11:39.566 INFO:tasks.rados.rados.0.smithi142.stdout:239: writing smithi14230322-50 from 1146880 to 1703936 tid 3 2024-04-18T23:11:39.566 INFO:tasks.rados.rados.0.smithi142.stdout:240: copy_from oid 49 from oid 4 current snap is 21 2024-04-18T23:11:39.566 INFO:tasks.rados.rados.0.smithi142.stdout:241: snap_create 2024-04-18T23:11:39.575 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:39.574+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:39.645 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:39.645 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:39.666 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:39.666 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:39.670 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:39.671 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:39.676 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:39.674+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:39.680 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:11:39.716 INFO:tasks.rados.rados.0.smithi142.stdout:232: finishing write tid 5 to smithi14230322-20 2024-04-18T23:11:39.716 INFO:tasks.rados.rados.0.smithi142.stdout:232: oid 20 updating version 0 to 32 2024-04-18T23:11:39.716 INFO:tasks.rados.rados.0.smithi142.stdout:232: oid 20 updating version 32 to 33 2024-04-18T23:11:39.717 INFO:tasks.rados.rados.0.smithi142.stdout:232: oid 20 updating version 33 to 34 2024-04-18T23:11:39.717 INFO:tasks.rados.rados.0.smithi142.stdout:232: oid 20 updating version 34 to 35 2024-04-18T23:11:39.717 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 20 v 35 (ObjNum 95 snap 21 seq_num 95) dirty exists 2024-04-18T23:11:39.717 INFO:tasks.rados.rados.0.smithi142.stdout:232: left oid 20 (ObjNum 95 snap 21 seq_num 95) 2024-04-18T23:11:39.739 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:39.760 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:39.761 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:39.777 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:39.775+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:39.806 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:39.807 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:39.822 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:11:39.829 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 48 v 74 (ObjNum 10 snap 0 seq_num 10) dirty exists 2024-04-18T23:11:39.856 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:39.857 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:39.878 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:39.874+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:39.880 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:39.880 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:39.900 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:39.901 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:39.927 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:39.927 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:39.928 INFO:tasks.rados.rados.0.smithi142.stdout:238: finishing write tid 1 to smithi14230322-11 2024-04-18T23:11:39.929 INFO:tasks.rados.rados.0.smithi142.stdout:238: finishing write tid 2 to smithi14230322-11 2024-04-18T23:11:39.929 INFO:tasks.rados.rados.0.smithi142.stdout:238: finishing write tid 3 to smithi14230322-11 2024-04-18T23:11:39.930 INFO:tasks.rados.rados.0.smithi142.stdout:238: finishing write tid 4 to smithi14230322-11 2024-04-18T23:11:39.931 INFO:tasks.rados.rados.0.smithi142.stdout:238: finishing write tid 5 to smithi14230322-11 2024-04-18T23:11:39.941 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:39.942 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:39.951 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:39.952 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:39.956 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:11:39.979 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:39.975+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:40.020 INFO:tasks.rados.rados.0.smithi142.stdout:240: finishing copy_from racing read to smithi14230322-49 2024-04-18T23:11:40.021 INFO:tasks.rados.rados.0.smithi142.stdout:239: finishing write tid 1 to smithi14230322-50 2024-04-18T23:11:40.033 INFO:tasks.rados.rados.0.smithi142.stdout:238: finishing write tid 6 to smithi14230322-11 2024-04-18T23:11:40.033 INFO:tasks.rados.rados.0.smithi142.stdout:238: oid 11 updating version 0 to 37 2024-04-18T23:11:40.033 INFO:tasks.rados.rados.0.smithi142.stdout:238: oid 11 updating version 37 to 41 2024-04-18T23:11:40.033 INFO:tasks.rados.rados.0.smithi142.stdout:238: oid 11 version 41 is already newer than 39 2024-04-18T23:11:40.033 INFO:tasks.rados.rados.0.smithi142.stdout:238: oid 11 version 41 is already newer than 38 2024-04-18T23:11:40.034 INFO:tasks.rados.rados.0.smithi142.stdout:238: oid 11 version 41 is already newer than 40 2024-04-18T23:11:40.034 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 11 v 41 (ObjNum 98 snap 21 seq_num 98) dirty exists 2024-04-18T23:11:40.034 INFO:tasks.rados.rados.0.smithi142.stdout:238: left oid 11 (ObjNum 98 snap 21 seq_num 98) 2024-04-18T23:11:40.078 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:40.078 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:40.080 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:40.079+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:40.093 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:11:40.122 INFO:tasks.rados.rados.0.smithi142.stdout:239: finishing write tid 2 to smithi14230322-50 2024-04-18T23:11:40.123 INFO:tasks.rados.rados.0.smithi142.stdout:239: finishing write tid 3 to smithi14230322-50 2024-04-18T23:11:40.123 INFO:tasks.rados.rados.0.smithi142.stdout:239: finishing write tid 4 to smithi14230322-50 2024-04-18T23:11:40.181 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:40.178+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:40.228 INFO:tasks.rados.rados.0.smithi142.stdout:239: finishing write tid 5 to smithi14230322-50 2024-04-18T23:11:40.228 INFO:tasks.rados.rados.0.smithi142.stdout:239: oid 50 updating version 0 to 116 2024-04-18T23:11:40.229 INFO:tasks.rados.rados.0.smithi142.stdout:239: oid 50 version 116 is already newer than 115 2024-04-18T23:11:40.229 INFO:tasks.rados.rados.0.smithi142.stdout:239: oid 50 updating version 116 to 118 2024-04-18T23:11:40.229 INFO:tasks.rados.rados.0.smithi142.stdout:239: oid 50 version 118 is already newer than 117 2024-04-18T23:11:40.229 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 50 v 118 (ObjNum 99 snap 21 seq_num 99) dirty exists 2024-04-18T23:11:40.229 INFO:tasks.rados.rados.0.smithi142.stdout:239: left oid 50 (ObjNum 99 snap 21 seq_num 99) 2024-04-18T23:11:40.261 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:40.262 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:40.276 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:11:40.322 INFO:tasks.rados.rados.0.smithi142.stdout:240: finishing copy_from to smithi14230322-49 2024-04-18T23:11:40.322 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 49 v 2 (ObjNum 3 snap 0 seq_num 3) dirty exists 2024-04-18T23:11:40.326 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:40.327+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:40.383 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:40.383+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:40.398 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:40.398 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:40.413 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:11:40.499 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:40.499+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:40.571 INFO:tasks.rados.rados.0.smithi142.stdout:232: done (5 left) 2024-04-18T23:11:40.571 INFO:tasks.rados.rados.0.smithi142.stdout:236: done (4 left) 2024-04-18T23:11:40.571 INFO:tasks.rados.rados.0.smithi142.stdout:238: done (3 left) 2024-04-18T23:11:40.571 INFO:tasks.rados.rados.0.smithi142.stdout:239: done (2 left) 2024-04-18T23:11:40.571 INFO:tasks.rados.rados.0.smithi142.stdout:240: done (1 left) 2024-04-18T23:11:40.571 INFO:tasks.rados.rados.0.smithi142.stdout:241: done (0 left) 2024-04-18T23:11:40.571 INFO:tasks.rados.rados.0.smithi142.stdout:242: snap_create 2024-04-18T23:11:40.571 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.3 to 1 (10000) 2024-04-18T23:11:40.584 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:40.584 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:40.585 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:40.582+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:40.594 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-18T23:11:40.598 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:11:40.686 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:40.683+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:40.732 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:40.732 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:40.746 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:11:40.790 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:40.790+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:40.870 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:40.870 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:40.884 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:11:40.887 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:40.887+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:40.988 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:40.986+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:41.054 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:41.054 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:41.068 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:11:41.089 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:41.086+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:41.190 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:41.187+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:41.233 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:41.234 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:41.247 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:11:41.291 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:41.291+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:41.371 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:41.371 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:41.385 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:11:41.391 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:41.391+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:41.492 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:41.491+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:41.508 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:41.508 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:41.522 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:11:41.546 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T23:11:41.577 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.4 to 1 (10000) 2024-04-18T23:11:41.578 INFO:tasks.rados.rados.0.smithi142.stdout:242: done (0 left) 2024-04-18T23:11:41.578 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 17 current snap is 23 2024-04-18T23:11:41.578 INFO:tasks.rados.rados.0.smithi142.stdout:243: seq_num 100 ranges {0=524288,524288=753664,1277952=409600} 2024-04-18T23:11:41.585 INFO:tasks.rados.rados.0.smithi142.stdout:243: writing smithi14230322-17 from 0 to 524288 tid 1 2024-04-18T23:11:41.593 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:41.591+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:41.598 INFO:tasks.rados.rados.0.smithi142.stdout:243: writing smithi14230322-17 from 524288 to 1277952 tid 2 2024-04-18T23:11:41.602 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-18T23:11:41.605 INFO:tasks.rados.rados.0.smithi142.stdout:243: writing smithi14230322-17 from 1277952 to 1687552 tid 3 2024-04-18T23:11:41.605 INFO:tasks.rados.rados.0.smithi142.stdout:244: rollback oid 8 current snap is 23 2024-04-18T23:11:41.605 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 8 to 21 2024-04-18T23:11:41.605 INFO:tasks.rados.rados.0.smithi142.stdout:245: rmattr oid 26 current snap is 23 2024-04-18T23:11:41.605 INFO:tasks.rados.rados.0.smithi142.stdout:245: done (2 left) 2024-04-18T23:11:41.606 INFO:tasks.rados.rados.0.smithi142.stdout:246: delete oid 2 current snap is 23 2024-04-18T23:11:41.642 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:41.642 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:41.656 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:11:41.693 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:41.690+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:41.708 INFO:tasks.rados.rados.0.smithi142.stdout:243: finishing write tid 1 to smithi14230322-17 2024-04-18T23:11:41.777 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:41.777 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:41.791 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:11:41.794 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:41.791+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:41.809 INFO:tasks.rados.rados.0.smithi142.stdout:243: finishing write tid 2 to smithi14230322-17 2024-04-18T23:11:41.894 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:41.890+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:41.909 INFO:tasks.rados.rados.0.smithi142.stdout:244: finishing rollback tid 0 to smithi14230322-8 2024-04-18T23:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:41.916 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:41.930 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:11:41.969 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:42.016 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:42.014+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:42.057 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:42.057 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:42.072 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:11:42.098 INFO:tasks.rados.rados.0.smithi142.stdout:243: finishing write tid 3 to smithi14230322-17 2024-04-18T23:11:42.098 INFO:tasks.rados.rados.0.smithi142.stdout:243: finishing write tid 4 to smithi14230322-17 2024-04-18T23:11:42.113 INFO:tasks.rados.rados.0.smithi142.stdout:244: finishing rollback tid 1 to smithi14230322-8 2024-04-18T23:11:42.113 INFO:tasks.rados.rados.0.smithi142.stdout:244: finishing rollback tid 2 to smithi14230322-8 2024-04-18T23:11:42.113 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 8 v 47 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-18T23:11:42.114 INFO:tasks.rados.rados.0.smithi142.stdout:244: done (2 left) 2024-04-18T23:11:42.114 INFO:tasks.rados.rados.0.smithi142.stdout:246: done (1 left) 2024-04-18T23:11:42.114 INFO:tasks.rados.rados.0.smithi142.stdout:247: rmattr oid 10 current snap is 23 2024-04-18T23:11:42.114 INFO:tasks.rados.rados.0.smithi142.stdout:247: done (1 left) 2024-04-18T23:11:42.114 INFO:tasks.rados.rados.0.smithi142.stdout:248: snap_create 2024-04-18T23:11:42.118 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:42.117+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:42.137 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:42.137 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:42.160 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:42.161 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:42.184 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:42.184 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:42.196 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:42.196 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:42.196 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:42.195+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:42.208 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:42.209 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:42.211 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:11:42.233 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:42.234 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:42.257 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:42.257 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:42.281 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:42.282 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:42.297 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:42.295+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:42.302 INFO:tasks.rados.rados.0.smithi142.stdout:243: finishing write tid 5 to smithi14230322-17 2024-04-18T23:11:42.302 INFO:tasks.rados.rados.0.smithi142.stdout:243: oid 17 updating version 0 to 58 2024-04-18T23:11:42.302 INFO:tasks.rados.rados.0.smithi142.stdout:243: oid 17 updating version 58 to 60 2024-04-18T23:11:42.302 INFO:tasks.rados.rados.0.smithi142.stdout:243: oid 17 version 60 is already newer than 59 2024-04-18T23:11:42.302 INFO:tasks.rados.rados.0.smithi142.stdout:243: oid 17 version 60 is already newer than 57 2024-04-18T23:11:42.302 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 17 v 60 (ObjNum 100 snap 23 seq_num 100) dirty exists 2024-04-18T23:11:42.302 INFO:tasks.rados.rados.0.smithi142.stdout:243: left oid 17 (ObjNum 100 snap 23 seq_num 100) 2024-04-18T23:11:42.305 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:42.325 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:42.325 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:42.348 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:42.348 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:42.371 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:42.371 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:42.379 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:42.379 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:42.393 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:11:42.393 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:42.394 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:42.414 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:42.414 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:42.436 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:42.437 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:42.442 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:42.443+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:42.458 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:42.459 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:42.480 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:42.480 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:42.498 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:42.497+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:42.527 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:42.527 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:42.527 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:42.527 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:42.541 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:11:42.589 INFO:tasks.rados.rados.0.smithi142.stdout:243: done (1 left) 2024-04-18T23:11:42.589 INFO:tasks.rados.rados.0.smithi142.stdout:248: done (0 left) 2024-04-18T23:11:42.589 INFO:tasks.rados.rados.0.smithi142.stdout:249: copy_from oid 48 from oid 47 current snap is 24 2024-04-18T23:11:42.589 INFO:tasks.rados.rados.0.smithi142.stdout:250: read oid 34 snap -1 2024-04-18T23:11:42.590 INFO:tasks.rados.rados.0.smithi142.stdout:250: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-18T23:11:42.590 INFO:tasks.rados.rados.0.smithi142.stdout:251: snap_create 2024-04-18T23:11:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-04-18T23:11:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.5 to 1 (10000) 2024-04-18T23:11:42.599 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:42.599+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:42.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-18T23:11:42.700 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:42.699+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:42.704 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:42.704 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:42.705 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:42.705 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:42.705 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:42.719 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:11:42.801 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:42.799+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:42.889 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:42.889 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:42.889 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:42.889 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:42.889 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(osd.5.21:2 3.as0 3:53611365:::smithi14230322-34:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.645176+0000", 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.242515658999999, 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20476035200000001, 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645176+0000", 2024-04-18T23:11:42.890 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645176+0000", 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645179+0000", 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.92e-06 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645190+0000", 2024-04-18T23:11:42.891 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0947e-05 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645212+0000", 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2075000000000001e-05 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645227+0000", 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4989e-05 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.892 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645346+0000", 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011948800000000001 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.645498+0000", 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015173899999999999 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.849936+0000", 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20443819399999999 2024-04-18T23:11:42.893 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 36/31 ECSubReadReply(tid=8, attrs_read=0))", 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.238828802, 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20110777299999999, 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.894 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.895 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648863+0000", 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648921+0000", 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.8091000000000003e-05 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.648981+0000", 2024-04-18T23:11:42.896 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0117000000000001e-05 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.849495+0000", 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.200514265 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.849971+0000", 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00047530000000000001 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.897 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 46/31 ECSubWrite(tid=41, reqid=client.4568.0:323, at_version=0'0, trim_to=0'0, roll_forward_to=46'23))", 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:11.424855+0000", 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 31.46283657, 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.1019087, 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424855+0000", 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.898 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424855+0000", 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424857+0000", 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2469999999999998e-06 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424910+0000", 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2784000000000001e-05 2024-04-18T23:11:42.899 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424957+0000", 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7141999999999997e-05 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.425023+0000", 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5814000000000004e-05 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525324+0000", 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100300527 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.900 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.525339+0000", 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5523999999999999e-05 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.526710+0000", 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001370511 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.526764+0000", 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4150999999999997e-05 2024-04-18T23:11:42.901 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s3 47/31 ECSubWrite(tid=79, reqid=client.4568.0:341, at_version=47'31, trim_to=0'0, roll_forward_to=45'29))", 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:12.829509+0000", 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 30.058182999, 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10224200899999999, 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.902 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829509+0000", 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829509+0000", 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829509+0000", 2024-04-18T23:11:42.903 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.3300000000000003e-07 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829546+0000", 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6777999999999999e-05 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.829574+0000", 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7645999999999999e-05 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.904 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.929675+0000", 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100101352 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.929815+0000", 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000140161 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.929836+0000", 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1015000000000001e-05 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.905 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.931716+0000", 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018800469999999999 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:12.931751+0000", 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4576999999999998e-05 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.906 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 51/31 ECSubWrite(tid=50, reqid=client.4568.0:389, at_version=51'31, trim_to=0'0, roll_forward_to=47'29))", 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.876486+0000", 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.011205897, 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10451489899999999, 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876486+0000", 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.907 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876486+0000", 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876486+0000", 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9999999999999997e-07 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876784+0000", 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029813000000000002 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.908 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876821+0000", 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6702999999999997e-05 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.976922+0000", 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100100598 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.977083+0000", 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016119800000000001 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.909 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.977113+0000", 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0273e-05 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980943+0000", 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0038295320000000001 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.981001+0000", 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7865000000000002e-05 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.910 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 61/31 ECSubWrite(tid=70, reqid=osd.7.0:70, at_version=61'48, trim_to=0'0, roll_forward_to=58'46))", 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:26.750469+0000", 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 16.137222913999999, 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10194142, 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750469+0000", 2024-04-18T23:11:42.911 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750469+0000", 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750469+0000", 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8299999999999998e-07 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.912 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750492+0000", 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3247999999999998e-05 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.750511+0000", 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8790000000000001e-05 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.850592+0000", 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100080628 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.913 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.850722+0000", 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013027199999999999 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.850739+0000", 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6515999999999999e-05 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.852244+0000", 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015051000000000001 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.914 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.852410+0000", 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016648300000000001 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:498 3.as0 3:5f3f0fa1:::smithi14230322-5:head [read 0~81920,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.529468+0000", 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.358223459, 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50906171700000002, 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:42.915 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:42.916 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:42.916 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:42.916 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 498 2024-04-18T23:11:42.916 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.916 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.916 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.916 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.916 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529468+0000", 2024-04-18T23:11:42.916 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529468+0000", 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529470+0000", 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2989999999999999e-06 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529482+0000", 2024-04-18T23:11:42.917 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1039000000000001e-05 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529526+0000", 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4184000000000003e-05 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529544+0000", 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8461000000000002e-05 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.918 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629759+0000", 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10021500999999999 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629766+0000", 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.294e-06 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.633240+0000", 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034736110000000001 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.919 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.633405+0000", 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000165353 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.038530+0000", 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40512446600000002 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:499 3.as0 3:5f3f0fa1:::smithi14230322-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.529590+0000", 2024-04-18T23:11:42.920 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.358102062, 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50955547999999995, 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 499 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529590+0000", 2024-04-18T23:11:42.921 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529590+0000", 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529590+0000", 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8999999999999995e-07 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529597+0000", 2024-04-18T23:11:42.922 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0230000000000004e-06 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529621+0000", 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3271000000000001e-05 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529631+0000", 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.075e-05 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629827+0000", 2024-04-18T23:11:42.923 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100195161 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629831+0000", 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6739999999999996e-06 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.635028+0000", 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.005196628 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.635198+0000", 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016980000000000001 2024-04-18T23:11:42.924 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.039145+0000", 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40394738299999999 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:500 3.as0 3:5f3f0fa1:::smithi14230322-5:head [read 0~81920,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.529647+0000", 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.358044244, 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.510256339, 2024-04-18T23:11:42.925 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 500 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529647+0000", 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.926 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529647+0000", 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529648+0000", 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4700000000000001e-07 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529652+0000", 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4740000000000001e-06 2024-04-18T23:11:42.927 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529658+0000", 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8009999999999998e-06 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529666+0000", 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3209999999999999e-06 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629919+0000", 2024-04-18T23:11:42.928 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100253156 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629933+0000", 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.379e-05 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.636127+0000", 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0061943980000000003 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.636263+0000", 2024-04-18T23:11:42.929 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013582700000000001 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.039904+0000", 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40364092499999998 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 66/31 ECSubReadReply(tid=15, attrs_read=0))", 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.638237+0000", 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.249454356999999, 2024-04-18T23:11:42.930 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.199117187, 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.638237+0000", 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.638237+0000", 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.931 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.638238+0000", 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.0999999999999997e-07 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.638274+0000", 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6165999999999999e-05 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.638294+0000", 2024-04-18T23:11:42.932 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9249000000000002e-05 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.738425+0000", 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10013111500000001 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.837277+0000", 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098852291999999994 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.933 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.837354+0000", 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.7454999999999996e-05 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 66/31 ECSubReadReply(tid=16, attrs_read=0))", 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.738472+0000", 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.149219964, 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19914532900000001, 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.934 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.738472+0000", 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.738472+0000", 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.935 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.738473+0000", 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.049e-06 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.738533+0000", 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9821999999999997e-05 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.738553+0000", 2024-04-18T23:11:42.936 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0934e-05 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.738565+0000", 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1086e-05 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.937537+0000", 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19897232400000001 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.937 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.937617+0000", 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0113999999999999e-05 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 66/31 ECSubReadReply(tid=14, attrs_read=0))", 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.831427+0000", 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.056264778999999, 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.207131753, 2024-04-18T23:11:42.938 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.831427+0000", 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.939 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.831427+0000", 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.831428+0000", 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.266e-06 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.831480+0000", 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.1999000000000001e-05 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.940 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.831503+0000", 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2387000000000001e-05 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.831514+0000", 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1841999999999999e-05 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.037833+0000", 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20631887199999999 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.941 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.038559+0000", 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00072538699999999997 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 66/31 ECSubReadReply(tid=15, attrs_read=0))", 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.931701+0000", 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.955990480000001, 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10746807799999999, 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.942 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.931701+0000", 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.931701+0000", 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.943 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.931704+0000", 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3659999999999999e-06 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.931759+0000", 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5426000000000001e-05 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.931809+0000", 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9568999999999999e-05 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.944 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.931830+0000", 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1855999999999999e-05 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.038666+0000", 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.106835281 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.039169+0000", 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00050358 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.945 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.as0 66/31 ECSubReadReply(tid=16, attrs_read=0))", 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.932227+0000", 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.955464881999999, 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.107706766, 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.946 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.932227+0000", 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.932227+0000", 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.932228+0000", 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3999999999999999e-06 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.947 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.932270+0000", 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1856e-05 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.932293+0000", 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2796e-05 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.932305+0000", 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2283000000000001e-05 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.948 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.039250+0000", 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.106944819 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.039934+0000", 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000683612 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.949 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 66/31 ECSubWrite(tid=84, reqid=client.4568.0:509, at_version=66'59, trim_to=0'0, roll_forward_to=65'57))", 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:32.116409+0000", 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.771282969, 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104529433, 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.116409+0000", 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.950 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.116409+0000", 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.116411+0000", 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5900000000000002e-06 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.117000+0000", 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000589103 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.951 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.117042+0000", 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1919000000000003e-05 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.117069+0000", 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.686e-05 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.952 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.217351+0000", 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100281632 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.217369+0000", 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.853e-05 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.220837+0000", 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0034673880000000001 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.953 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.220938+0000", 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000101411 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 71/31 ECSubWrite(tid=89, reqid=osd.7.0:89, at_version=71'65, trim_to=0'0, roll_forward_to=69'63))", 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:37.771396+0000", 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.1162960499999999, 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102597413, 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.954 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.771396+0000", 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.771396+0000", 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.771397+0000", 2024-04-18T23:11:42.955 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8470000000000001e-06 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.771442+0000", 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4746000000000002e-05 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.771473+0000", 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1186000000000003e-05 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.956 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.871644+0000", 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10017049 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.871816+0000", 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017176999999999999 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.871832+0000", 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6007000000000001e-05 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.873896+0000", 2024-04-18T23:11:42.957 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002064532 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.873993+0000", 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6835000000000004e-05 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.as0 71/31 ECSubWriteReply(tid=20, last_complete=71'8, committed=1, applied=1))", 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:37.874492+0000", 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.0131997840000002, 2024-04-18T23:11:42.958 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.110371833, 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.874492+0000", 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.874492+0000", 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.959 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.874494+0000", 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.734e-06 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.874505+0000", 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1892999999999999e-05 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.874535+0000", 2024-04-18T23:11:42.960 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9218999999999999e-05 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.974668+0000", 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10013369799999999 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.984805+0000", 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010136157 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.984864+0000", 2024-04-18T23:11:42.961 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9132000000000002e-05 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.ds1 72/31 ECSubWrite(tid=93, reqid=osd.7.0:93, at_version=72'71, trim_to=0'0, roll_forward_to=71'67))", 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.712260+0000", 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.1754318929999998, 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.202256877, 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.962 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.712260+0000", 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.712260+0000", 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.712262+0000", 2024-04-18T23:11:42.963 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.154e-06 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.712312+0000", 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0198999999999998e-05 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.712349+0000", 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6959000000000003e-05 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.812448+0000", 2024-04-18T23:11:42.964 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100098873 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.912705+0000", 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10025711 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.912722+0000", 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6900000000000001e-05 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.914452+0000", 2024-04-18T23:11:42.965 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0017300289999999999 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.914517+0000", 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4653000000000006e-05 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s2 72/67 ECSubWrite(tid=206, reqid=osd.13.0:205, at_version=0'0, trim_to=0'0, roll_forward_to=72'35))", 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.818024+0000", 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.0696680790000004, 2024-04-18T23:11:42.966 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.198673672, 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.818024+0000", 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.818024+0000", 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.967 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.818025+0000", 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6270000000000001e-06 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.818045+0000", 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0205999999999999e-05 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.818072+0000", 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.65e-05 2024-04-18T23:11:42.968 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.818087+0000", 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5463000000000001e-05 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.916503+0000", 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098415717 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.916529+0000", 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6342999999999998e-05 2024-04-18T23:11:42.969 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.016637+0000", 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10010717399999999 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.016697+0000", 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0642e-05 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.970 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:599 3.as0 3:53611365:::smithi14230322-34:head [read 0~557056,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:42.591484+0000", 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.29620720299999997, 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20487891999999999, 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 599 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:42.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591484+0000", 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591484+0000", 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591487+0000", 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3819999999999998e-06 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591507+0000", 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9919e-05 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591530+0000", 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3487999999999999e-05 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591541+0000", 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1144e-05 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591643+0000", 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000101721 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591651+0000", 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9980000000000003e-06 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.593496+0000", 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018447870000000001 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.974 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.593603+0000", 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010710899999999999 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.796363+0000", 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20276037199999999 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.975 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.976 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:42.976 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:42.976 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:42.977 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:11:42.978 INFO:tasks.rados.rados.0.smithi142.stdout:249: finishing copy_from racing read to smithi14230322-48 2024-04-18T23:11:42.978 INFO:tasks.rados.rados.0.smithi142.stdout:250: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-18T23:11:42.979 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:42.979+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:43.009 INFO:tasks.rados.rados.0.smithi142.stdout:249: finishing copy_from to smithi14230322-48 2024-04-18T23:11:43.009 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 48 v 14 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-18T23:11:43.078 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:43.077+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:43.102 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:43.102 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:43.102 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 76/31 ECSubWriteReply(tid=104, last_complete=76'82, committed=1, applied=1))", 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:43.012557+0000", 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.09123887, 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.091280049000000002, 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012557+0000", 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.103 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012557+0000", 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012558+0000", 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5700000000000001e-07 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012564+0000", 2024-04-18T23:11:43.104 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.4709999999999998e-06 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012579+0000", 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5216e-05 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012587+0000", 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2740000000000007e-06 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.105 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 76/31 ECSubWriteReply(tid=104, last_complete=76'82, committed=1, applied=1))", 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:43.012825+0000", 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.090970659999999995, 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.091122892999999996, 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012825+0000", 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.106 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012825+0000", 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012826+0000", 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1100000000000005e-07 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012832+0000", 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.3510000000000001e-06 2024-04-18T23:11:43.107 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012845+0000", 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3719e-05 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012856+0000", 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0932e-05 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.108 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 76/31 ECSubWriteReply(tid=104, last_complete=76'82, committed=1, applied=1))", 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:43.012867+0000", 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.090929394999999996, 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.091155953999999997, 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012867+0000", 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.109 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012867+0000", 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012867+0000", 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9100000000000004e-07 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012871+0000", 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1309999999999999e-06 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.110 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012883+0000", 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2016e-05 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012892+0000", 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1440000000000003e-06 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.111 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 76/31 ECSubWriteReply(tid=104, last_complete=76'82, committed=1, applied=1))", 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:43.012905+0000", 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.090890653000000002, 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.091206850000000006, 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012905+0000", 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.112 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012905+0000", 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012906+0000", 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.2699999999999999e-07 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012909+0000", 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4939999999999999e-06 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.113 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012921+0000", 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1620000000000001e-05 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012928+0000", 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.2830000000000004e-06 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.114 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.ds0 76/31 ECSubWriteReply(tid=104, last_complete=76'82, committed=1, applied=1))", 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:43.012941+0000", 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.090854853999999999, 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.091263537000000006, 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012941+0000", 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.115 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012941+0000", 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012942+0000", 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4999999999999998e-07 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012945+0000", 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4960000000000001e-06 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.116 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012957+0000", 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1311000000000001e-05 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.012969+0000", 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2422999999999999e-05 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 5 2024-04-18T23:11:43.117 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:43.118 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:11:43.179 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:43.174+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:43.278 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:43.278 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:43.278 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:43.279 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:43.279 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:43.280 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:43.278+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:43.292 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:11:43.381 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:43.377+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:43.418 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:43.418 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:43.418 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:43.418 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:43.418 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.418 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:186 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [rollback 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:43.418 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.646882+0000", 2024-04-18T23:11:43.418 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.768941980000001, 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40783060199999999, 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 186 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646882+0000", 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.419 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646882+0000", 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646884+0000", 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.513e-06 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646894+0000", 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7229999999999999e-06 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.420 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646913+0000", 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9769e-05 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.646929+0000", 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5586e-05 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.746864+0000", 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099935144000000004 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.421 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.747144+0000", 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028019299999999998 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.748787+0000", 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0016425890000000001 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752698+0000", 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0039113430000000003 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.422 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.054522+0000", 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30182420799999998 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.054713+0000", 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019053399999999999 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:187 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~0] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:43.423 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647001+0000", 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.768823113000003, 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40801147199999999, 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 187 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647001+0000", 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.424 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647001+0000", 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647002+0000", 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2270000000000001e-06 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647009+0000", 2024-04-18T23:11:43.425 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.8229999999999997e-06 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647033+0000", 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3646e-05 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647052+0000", 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9279e-05 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.749808+0000", 2024-04-18T23:11:43.426 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102756006 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.750075+0000", 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026714700000000002 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.751371+0000", 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001295631 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752960+0000", 2024-04-18T23:11:43.427 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015885680000000001 2024-04-18T23:11:43.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.428 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.054910+0000", 2024-04-18T23:11:43.428 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30195069600000002 2024-04-18T23:11:43.428 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.428 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.428 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.055013+0000", 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010244899999999999 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:202 3.ds0 3:ba4438ef:::smithi14230322-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.768748533, 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.726850518, 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.429 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 202 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.430 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647077+0000", 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.497e-06 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647086+0000", 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5490000000000006e-06 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647105+0000", 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8933e-05 2024-04-18T23:11:43.431 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647120+0000", 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4942e-05 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752033+0000", 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10491311 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752334+0000", 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030070900000000002 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.432 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.072768+0000", 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32043411399999999 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.075547+0000", 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027797260000000002 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.373671+0000", 2024-04-18T23:11:43.433 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29812367699999998 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.373926+0000", 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025526100000000001 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:203 3.ds0 3:ba4438ef:::smithi14230322-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647172+0000", 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.768652580999998, 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.42781979999999997, 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.434 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 203 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647172+0000", 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647172+0000", 2024-04-18T23:11:43.435 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647174+0000", 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8369999999999999e-06 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647180+0000", 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9529999999999996e-06 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647201+0000", 2024-04-18T23:11:43.436 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1501999999999998e-05 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647218+0000", 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.647e-05 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.753422+0000", 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10620457699999999 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.753567+0000", 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014483399999999999 2024-04-18T23:11:43.437 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.074716+0000", 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.321149236 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.074851+0000", 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000135258 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.074992+0000", 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014013299999999999 2024-04-18T23:11:43.438 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:318 3.ds0 3:ba4438ef:::smithi14230322-48:7 [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609705+0000", 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 32.806119285999998, 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40842472699999999, 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.439 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 318 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609705+0000", 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609705+0000", 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.440 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609707+0000", 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7e-06 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609725+0000", 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7992999999999999e-05 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609740+0000", 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5259000000000001e-05 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.441 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609757+0000", 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6806000000000001e-05 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609990+0000", 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000232721 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.610013+0000", 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3203000000000001e-05 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.442 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.712982+0000", 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102969296 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.713166+0000", 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000184217 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.018130+0000", 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30496353199999998 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.443 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:319 3.ds0 3:ba4438ef:::smithi14230322-48:7 [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609786+0000", 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 32.806038078999997, 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50925149999999997, 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 319 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.444 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609786+0000", 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609786+0000", 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609788+0000", 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3999999999999999e-06 2024-04-18T23:11:43.445 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609795+0000", 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3200000000000002e-06 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609813+0000", 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8493000000000001e-05 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609830+0000", 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6062e-05 2024-04-18T23:11:43.446 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.610105+0000", 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000275467 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.610126+0000", 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1262000000000001e-05 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.814838+0000", 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.204711587 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.447 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.815034+0000", 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000196579 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119038+0000", 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30400333000000002 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:323 3.ds0 3:b8bdcfdb:::smithi14230322-44:head [delete] snapc 8=[8,7,4] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:11:43.448 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 32.805963427999998, 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81471389100000002, 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 323 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.449 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609863+0000", 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6360000000000001e-06 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609869+0000", 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4699999999999999e-06 2024-04-18T23:11:43.450 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609890+0000", 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0496000000000002e-05 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609905+0000", 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5539000000000001e-05 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.614202+0000", 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042970489999999998 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.451 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.614219+0000", 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7373000000000001e-05 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119137+0000", 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50491712899999996 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119293+0000", 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015688099999999999 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.452 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120709+0000", 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014158669999999999 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.122707+0000", 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019980140000000002 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424382+0000", 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30167486399999999 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.453 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424575+0000", 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000192573 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:389 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~458752] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.672684+0000", 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.743140476000001, 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71125476700000001, 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.454 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 389 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.672684+0000", 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.672684+0000", 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.455 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.672687+0000", 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.439e-06 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.673226+0000", 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00053900300000000003 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.673261+0000", 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5169000000000002e-05 2024-04-18T23:11:43.456 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.673279+0000", 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7353000000000001e-05 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.676118+0000", 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002839453 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.676134+0000", 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5489999999999999e-05 2024-04-18T23:11:43.457 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.874209+0000", 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.198074958 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.874392+0000", 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018306999999999999 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.458 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876303+0000", 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001910878 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.881915+0000", 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0056125589999999996 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.383786+0000", 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50187064000000003 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.459 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.383939+0000", 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000152755 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:390 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~475136] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.675363+0000", 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.740461251999999, 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.80904401299999995, 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.460 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 390 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675363+0000", 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675363+0000", 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675365+0000", 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1569999999999998e-06 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675936+0000", 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00057051499999999995 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675968+0000", 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1779000000000002e-05 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675983+0000", 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5313000000000001e-05 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.873642+0000", 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.197659001 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.873655+0000", 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3073e-05 2024-04-18T23:11:43.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.877399+0000", 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0037437019999999998 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.877612+0000", 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021331900000000001 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.879219+0000", 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001607266 2024-04-18T23:11:43.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.881917+0000", 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026975240000000002 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484280+0000", 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60236371499999997 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484407+0000", 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000126649 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.465 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:391 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~294912] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.677038+0000", 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.738786464, 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.80775349699999999, 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 391 2024-04-18T23:11:43.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677038+0000", 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677038+0000", 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677040+0000", 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0059999999999999e-06 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677406+0000", 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00036561500000000002 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677440+0000", 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4972999999999999e-05 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777555+0000", 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100114034 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.880210+0000", 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10265596 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.880389+0000", 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017841299999999999 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.881177+0000", 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00078838699999999999 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.183072+0000", 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30189508599999998 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484661+0000", 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30158817300000001 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484791+0000", 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013085000000000001 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:392 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [setxattr _header (57) in=7b] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.777603+0000", 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.638221660999999, 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.70764648600000002, 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.471 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 392 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777603+0000", 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777603+0000", 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777605+0000", 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0439999999999998e-06 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777618+0000", 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3169e-05 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777650+0000", 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2492e-05 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777674+0000", 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3896e-05 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980335+0000", 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.202661059 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980638+0000", 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030296400000000002 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.981741+0000", 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011024609999999999 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.183074+0000", 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20133356599999999 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.475 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.485059+0000", 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30198478499999998 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.485249+0000", 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019005000000000001 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:393 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.777705+0000", 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.638119617000001, 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0113445000000001, 2024-04-18T23:11:43.476 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 393 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777705+0000", 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777705+0000", 2024-04-18T23:11:43.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777706+0000", 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.626e-06 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777716+0000", 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0146000000000001e-05 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777742+0000", 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5236e-05 2024-04-18T23:11:43.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777762+0000", 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9879999999999999e-05 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.982379+0000", 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20461718700000001 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.982705+0000", 2024-04-18T23:11:43.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032662899999999999 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.486129+0000", 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50342405499999998 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.486297+0000", 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016788799999999999 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.789049+0000", 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30275185300000002 2024-04-18T23:11:43.480 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:426 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [copy-from ver 16] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:21.537160+0000", 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.878663895999999, 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.52374796700000004, 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 426 2024-04-18T23:11:43.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537160+0000", 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537160+0000", 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537163+0000", 2024-04-18T23:11:43.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3750000000000001e-06 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537176+0000", 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2755000000000001e-05 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537201+0000", 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5451000000000001e-05 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537212+0000", 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0614e-05 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537295+0000", 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3707000000000003e-05 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537302+0000", 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0729999999999999e-06 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.741218+0000", 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20391555 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.741402+0000", 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018423199999999999 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.953934+0000", 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21253224000000001 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.058114+0000", 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104179409 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.060716+0000", 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026026000000000001 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.060908+0000", 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019196099999999999 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:427 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:21.537234+0000", 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.878590575, 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.52066839700000001, 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.486 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 427 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537234+0000", 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.487 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537234+0000", 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537234+0000", 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2300000000000002e-07 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537237+0000", 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4140000000000001e-06 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537242+0000", 2024-04-18T23:11:43.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6500000000000001e-06 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537246+0000", 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6839999999999998e-06 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537413+0000", 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016680199999999999 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537424+0000", 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1379999999999999e-05 2024-04-18T23:11:43.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.741750+0000", 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20432588900000001 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.741914+0000", 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016375299999999999 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.057567+0000", 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31565273700000002 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.057722+0000", 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000155883 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.057902+0000", 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017978199999999999 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:430 3.ds0 3:ba4438ef:::smithi14230322-48:b [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:21.537331+0000", 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.878493130999999, 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40852623700000001, 2024-04-18T23:11:43.491 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 430 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537331+0000", 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537331+0000", 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537331+0000", 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5899999999999998e-07 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537333+0000", 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.545e-06 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537336+0000", 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3610000000000001e-06 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537339+0000", 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7999999999999999e-06 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.637533+0000", 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10019347000000001 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.637543+0000", 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0815999999999999e-05 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.745610+0000", 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.1080663 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.745763+0000", 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000153325 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.945857+0000", 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.200094361 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.495 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:502 3.ds0 3:ba4438ef:::smithi14230322-48:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.529587+0000", 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.886237102000001, 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.58018316299999995, 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.496 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 502 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529587+0000", 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529587+0000", 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529588+0000", 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.009e-06 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529594+0000", 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.254e-06 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529607+0000", 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3730999999999999e-05 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529616+0000", 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8570000000000001e-06 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.531417+0000", 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018004500000000001 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.531690+0000", 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00027340099999999999 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.109770+0000", 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.57808046099999999 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:503 3.ds0 3:ba4438ef:::smithi14230322-48:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.529671+0000", 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.886153811, 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.58061668499999997, 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.500 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 503 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529671+0000", 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529671+0000", 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529671+0000", 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4900000000000005e-07 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529676+0000", 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6779999999999999e-06 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529687+0000", 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1404e-05 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629836+0000", 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100148557 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.633504+0000", 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036684729999999998 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.633773+0000", 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026893700000000002 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.110287+0000", 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.47651378700000002 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:509 3.ds0 3:bb40a4d8:::smithi14230322-41:head [copy-from ver 20] snapc 15=[15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.629899+0000", 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.785925528, 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.62332154699999998, 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.504 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 509 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629899+0000", 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629899+0000", 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629902+0000", 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7329999999999999e-06 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629919+0000", 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.734e-05 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629948+0000", 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8782999999999999e-05 2024-04-18T23:11:43.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629961+0000", 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3349999999999999e-05 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.735207+0000", 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105245482 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.735530+0000", 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032370599999999999 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.116389+0000", 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38085889099999998 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.120910+0000", 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0045208920000000003 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.252995+0000", 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13208513499999999 2024-04-18T23:11:43.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.253220+0000", 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022523500000000001 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:510 3.ds0 3:bb40a4d8:::smithi14230322-41:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.629979+0000", 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.785845800000001, 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49023087100000001, 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.509 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 510 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629979+0000", 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629979+0000", 2024-04-18T23:11:43.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629980+0000", 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.06e-06 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629984+0000", 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6430000000000003e-06 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629995+0000", 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1060000000000001e-05 2024-04-18T23:11:43.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.730164+0000", 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100168985 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.736017+0000", 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0058526969999999996 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.736111+0000", 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3894000000000006e-05 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.119824+0000", 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.383712834 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.120018+0000", 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000194465 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.120209+0000", 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019123299999999999 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.513 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:597 3.ds0 3:ba4438ef:::smithi14230322-48:head [copy-from ver 14] snapc 19=[19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e76)", 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:42.591333+0000", 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.82449093600000001, 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.41960652799999998, 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 597 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.514 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591333+0000", 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591333+0000", 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591337+0000", 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4869999999999998e-06 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.515 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591351+0000", 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3675e-05 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591377+0000", 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6611e-05 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591392+0000", 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4759999999999999e-05 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591569+0000", 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000176882 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.591594+0000", 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4870000000000001e-05 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.792131+0000", 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20053768699999999 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.792231+0000", 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.9481000000000005e-05 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.904125+0000", 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.111894165 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.907317+0000", 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031918039999999999 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.010794+0000", 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103477152 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.010940+0000", 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014595400000000001 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.519 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:43.521 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:11:43.521 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:43.519+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:43.598 INFO:tasks.rados.rados.0.smithi142.stdout:249: done (2 left) 2024-04-18T23:11:43.598 INFO:tasks.rados.rados.0.smithi142.stdout:250: done (1 left) 2024-04-18T23:11:43.598 INFO:tasks.rados.rados.0.smithi142.stdout:251: done (0 left) 2024-04-18T23:11:43.598 INFO:tasks.rados.rados.0.smithi142.stdout:252: read oid 47 snap -1 2024-04-18T23:11:43.599 INFO:tasks.rados.rados.0.smithi142.stdout:252: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-18T23:11:43.599 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 45 current snap is 25 2024-04-18T23:11:43.599 INFO:tasks.rados.rados.0.smithi142.stdout:253: seq_num 102 ranges {753664=557056,1310720=638976,1949696=524288} 2024-04-18T23:11:43.599 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.6 to 1 (10000) 2024-04-18T23:11:43.609 INFO:tasks.rados.rados.0.smithi142.stdout:253: writing smithi14230322-45 from 753664 to 1310720 tid 1 2024-04-18T23:11:43.614 INFO:tasks.rados.rados.0.smithi142.stdout:253: writing smithi14230322-45 from 1310720 to 1949696 tid 2 2024-04-18T23:11:43.619 INFO:tasks.rados.rados.0.smithi142.stdout:253: writing smithi14230322-45 from 1949696 to 2473984 tid 3 2024-04-18T23:11:43.619 INFO:tasks.rados.rados.0.smithi142.stdout:254: read oid 49 snap 19 2024-04-18T23:11:43.619 INFO:tasks.rados.rados.0.smithi142.stdout:254: expect (ObjNum 64 snap 7 seq_num 64) 2024-04-18T23:11:43.619 INFO:tasks.rados.rados.0.smithi142.stdout:255: read oid 28 snap -1 2024-04-18T23:11:43.619 INFO:tasks.rados.rados.0.smithi142.stdout:255: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:43.620 INFO:tasks.rados.rados.0.smithi142.stdout:256: read oid 11 snap -1 2024-04-18T23:11:43.620 INFO:tasks.rados.rados.0.smithi142.stdout:256: expect (ObjNum 98 snap 21 seq_num 98) 2024-04-18T23:11:43.620 INFO:tasks.rados.rados.0.smithi142.stdout:257: read oid 5 snap 23 2024-04-18T23:11:43.620 INFO:tasks.rados.rados.0.smithi142.stdout:257: expect deleted 2024-04-18T23:11:43.620 INFO:tasks.rados.rados.0.smithi142.stdout:258: setattr oid 48 current snap is 25 2024-04-18T23:11:43.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-18T23:11:43.622 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:43.618+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:43.623 INFO:tasks.rados.rados.0.smithi142.stdout:259: rollback oid 14 current snap is 25 2024-04-18T23:11:43.623 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 14 to 15 2024-04-18T23:11:43.623 INFO:tasks.rados.rados.0.smithi142.stdout:260: delete oid 18 current snap is 25 2024-04-18T23:11:43.690 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:43.690 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:43.690 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:43.690 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:43.704 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:11:43.722 INFO:tasks.rados.rados.0.smithi142.stdout:255: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:11:43.722 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:43.719+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:43.824 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:43.823+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:43.825 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 48 v 84 (ObjNum 46 snap 0 seq_num 46) dirty exists 2024-04-18T23:11:43.827 INFO:tasks.rados.rados.0.smithi142.stdout:255: done (8 left) 2024-04-18T23:11:43.828 INFO:tasks.rados.rados.0.smithi142.stdout:257: done (7 left) 2024-04-18T23:11:43.828 INFO:tasks.rados.rados.0.smithi142.stdout:258: done (6 left) 2024-04-18T23:11:43.828 INFO:tasks.rados.rados.0.smithi142.stdout:260: done (5 left) 2024-04-18T23:11:43.828 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 19 current snap is 25 2024-04-18T23:11:43.828 INFO:tasks.rados.rados.0.smithi142.stdout:261: seq_num 104 ranges {0=589824,589824=540672,1130496=622592} 2024-04-18T23:11:43.836 INFO:tasks.rados.rados.0.smithi142.stdout:261: writing smithi14230322-19 from 0 to 589824 tid 1 2024-04-18T23:11:43.843 INFO:tasks.rados.rados.0.smithi142.stdout:261: writing smithi14230322-19 from 589824 to 1130496 tid 2 2024-04-18T23:11:43.846 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:43.846 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:43.846 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:43.846 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:43.846 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:43.852 INFO:tasks.rados.rados.0.smithi142.stdout:261: writing smithi14230322-19 from 1130496 to 1753088 tid 3 2024-04-18T23:11:43.852 INFO:tasks.rados.rados.0.smithi142.stdout:262: setattr oid 25 current snap is 25 2024-04-18T23:11:43.857 INFO:tasks.rados.rados.0.smithi142.stdout:263: setattr oid 15 current snap is 25 2024-04-18T23:11:43.860 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:11:43.863 INFO:tasks.rados.rados.0.smithi142.stdout:264: snap_remove snap 18 2024-04-18T23:11:43.925 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:43.922+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:43.988 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:192 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.426055484999999, 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30501581700000002, 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.989 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 192 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560439+0000", 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6029999999999999e-06 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560450+0000", 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0835999999999999e-05 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560478+0000", 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7619000000000001e-05 2024-04-18T23:11:43.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560489+0000", 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0998999999999999e-05 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560680+0000", 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019082800000000001 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560688+0000", 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2260000000000005e-06 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.992 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662105+0000", 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10141672 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662220+0000", 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115872 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.865454+0000", 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20323311399999999 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.993 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:193 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.425946740000001, 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30581751899999998, 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 193 2024-04-18T23:11:43.994 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:43.995 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560548+0000", 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560554+0000", 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2230000000000004e-06 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560566+0000", 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2622999999999999e-05 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560575+0000", 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1589999999999993e-06 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563654+0000", 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030794709999999999 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563665+0000", 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0708e-05 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:43.997 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662682+0000", 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099017665000000005 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.663181+0000", 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049841999999999996 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.866364+0000", 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20318328699999999 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:43.998 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:210 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.425849132000003, 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.41278097800000002, 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 210 2024-04-18T23:11:43.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560645+0000", 2024-04-18T23:11:44.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1829999999999999e-06 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560652+0000", 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3500000000000002e-06 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560669+0000", 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6833e-05 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560682+0000", 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3585e-05 2024-04-18T23:11:44.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560856+0000", 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017400200000000001 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560874+0000", 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8386e-05 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767285+0000", 2024-04-18T23:11:44.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20641079700000001 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767423+0000", 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013740399999999999 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.973425+0000", 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20600243800000001 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.003 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:211 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.425760836000002, 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51049911299999995, 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.004 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 211 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.005 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560734+0000", 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-06 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560741+0000", 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1559999999999998e-06 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560758+0000", 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7382999999999999e-05 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560775+0000", 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7051999999999999e-05 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560940+0000", 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164659 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560949+0000", 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7630000000000006e-06 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767943+0000", 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.206994276 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.768107+0000", 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164358 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.071232+0000", 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30312427600000003 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:359 3.cs0 3:35606aaf:::smithi14230322-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 30.790244886, 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.36957785599999998, 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 359 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.009 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196249+0000", 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5700000000000002e-07 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196253+0000", 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8979999999999998e-06 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196271+0000", 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8270999999999998e-05 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196280+0000", 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.85e-06 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196650+0000", 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00037011599999999998 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196698+0000", 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8060999999999999e-05 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565575+0000", 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.36887705999999998 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565695+0000", 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119663 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565826+0000", 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013118 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:384 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~442368] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:44.013 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.321951582000001, 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51544569399999995, 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 384 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664544+0000", 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6240000000000002e-06 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665037+0000", 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049266399999999997 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665070+0000", 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2963000000000003e-05 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665089+0000", 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8672e-05 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665233+0000", 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014389900000000001 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:44.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665245+0000", 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2446000000000001e-05 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769742+0000", 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104497225 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.770038+0000", 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029553099999999997 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.772939+0000", 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029013009999999998 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781282+0000", 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0083427859999999996 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.179880+0000", 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.398598591 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.179987+0000", 2024-04-18T23:11:44.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010699199999999999 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:385 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~720896] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.317877957, 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61243552499999998, 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.019 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 385 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668618+0000", 2024-04-18T23:11:44.020 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5129999999999999e-06 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669576+0000", 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000957726 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669605+0000", 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9566000000000001e-05 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669622+0000", 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6409000000000002e-05 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.021 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.775317+0000", 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105695331 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.775501+0000", 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018403000000000001 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777261+0000", 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001759652 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781284+0000", 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040233049999999996 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.280844+0000", 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49956001900000002 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281051+0000", 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020697399999999999 2024-04-18T23:11:44.023 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:386 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~49152] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.316840887000001, 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61210873300000002, 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 386 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.024 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669654+0000", 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.297e-06 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669730+0000", 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6474e-05 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669748+0000", 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7598999999999999e-05 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769883+0000", 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100135671 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.778786+0000", 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0089029639999999993 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.779085+0000", 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029830699999999999 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.780258+0000", 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011728070000000001 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783446+0000", 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031888179999999999 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281593+0000", 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49814646800000001 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281761+0000", 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016832800000000001 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.028 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:387 3.fs0 3:f47c3018:::smithi14230322-43:head [setxattr _header (57) in=7b] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.216570331, 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51243170900000001, 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 387 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769925+0000", 2024-04-18T23:11:44.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7799999999999999e-06 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769934+0000", 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6460000000000005e-06 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769956+0000", 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1216000000000002e-05 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769967+0000", 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-05 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781530+0000", 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.01156212 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781688+0000", 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015813200000000001 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.782502+0000", 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00081454100000000001 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783689+0000", 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001186552 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.282098+0000", 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49840956600000003 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.282355+0000", 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025625600000000001 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:388 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.216512903000002, 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81817408400000002, 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.034 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 388 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:11:44.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769981+0000", 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6400000000000001e-07 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769985+0000", 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7689999999999998e-06 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.036 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769996+0000", 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.075e-05 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.770003+0000", 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5769999999999996e-06 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783045+0000", 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013041308999999999 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:44.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783303+0000", 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025812900000000001 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.283544+0000", 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50024145900000005 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.283825+0000", 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028086099999999998 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.588154+0000", 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30432946599999999 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.fs0 51/31 ECSubReadReply(tid=28, attrs_read=0))", 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.701401436000001, 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30201136499999998, 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285093+0000", 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.08e-06 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.040 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285111+0000", 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8272000000000001e-05 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285129+0000", 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7706999999999998e-05 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285140+0000", 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1472e-05 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587038+0000", 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30189753600000002 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587103+0000", 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5297999999999997e-05 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.fs0 51/31 ECSubRead(tid=28, to_read={3:f47c3018:::smithi14230322-43:head=0,4096,0}, subchunks={3:f47c3018:::smithi14230322-43:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.700801232, 2024-04-18T23:11:44.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30165329400000002, 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285725+0000", 2024-04-18T23:11:44.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3365999999999999e-05 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.385871+0000", 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100145397 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587204+0000", 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.201332972 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587345+0000", 2024-04-18T23:11:44.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000141559 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 61/31 ECSubWriteReply(tid=39, last_complete=61'15, committed=1, applied=1))", 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:26.653658+0000", 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 17.332835421999999, 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50232716700000002, 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653658+0000", 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653658+0000", 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653660+0000", 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2170000000000001e-06 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653670+0000", 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0183000000000001e-05 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653689+0000", 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9026e-05 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653701+0000", 2024-04-18T23:11:44.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1922999999999999e-05 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:27.154982+0000", 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50128102200000002 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:27.155985+0000", 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001002796 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.049 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 61/31 ECSubWriteReply(tid=39, last_complete=61'15, committed=1, applied=1))", 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:26.653748+0000", 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 17.332745770999999, 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.301973926, 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653748+0000", 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653748+0000", 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653749+0000", 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2270000000000001e-06 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653755+0000", 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3409999999999999e-06 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653771+0000", 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6300999999999999e-05 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653786+0000", 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4236999999999999e-05 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.955692+0000", 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30190629699999999 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.955721+0000", 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9522999999999998e-05 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:561 3.cs0 3:35606aaf:::smithi14230322-20:head [append 0~442368] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.524261+0000", 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.4622327630000003, 2024-04-18T23:11:44.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31889435799999999, 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 561 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524261+0000", 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524261+0000", 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524263+0000", 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8040000000000002e-06 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524799+0000", 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00053593200000000003 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524834+0000", 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4693000000000001e-05 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524852+0000", 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7612999999999999e-05 2024-04-18T23:11:44.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.525052+0000", 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020047899999999999 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.525066+0000", 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3937000000000001e-05 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.629867+0000", 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10480112 2024-04-18T23:11:44.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.630105+0000", 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00023814499999999999 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.631631+0000", 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015257669999999999 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.641942+0000", 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010310784 2024-04-18T23:11:44.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.842982+0000", 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20104003100000001 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.843155+0000", 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000173053 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:562 3.cs0 3:35606aaf:::smithi14230322-20:head [append 0~540672] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.531229+0000", 2024-04-18T23:11:44.059 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.4552644429999999, 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61875124800000003, 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 562 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531229+0000", 2024-04-18T23:11:44.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531229+0000", 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531231+0000", 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0530000000000001e-06 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531984+0000", 2024-04-18T23:11:44.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00075264199999999998 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.532012+0000", 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8932999999999999e-05 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.532029+0000", 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6203000000000001e-05 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.638598+0000", 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.106569199 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.638913+0000", 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031543000000000001 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.640689+0000", 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001775715 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.742333+0000", 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101644006 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.149762+0000", 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40742915800000001 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.149980+0000", 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021790899999999999 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.064 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:563 3.cs0 3:35606aaf:::smithi14230322-20:head [append 0~622592] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.538968+0000", 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.4475252090000001, 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.68520978200000004, 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 563 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.538968+0000", 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.065 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.538968+0000", 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.538970+0000", 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0559999999999999e-06 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539699+0000", 2024-04-18T23:11:44.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00072919900000000004 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539729+0000", 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9899999999999998e-05 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539744+0000", 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4708999999999999e-05 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.943419+0000", 2024-04-18T23:11:44.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.403674637 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.943605+0000", 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018626 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.945303+0000", 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001697893 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148746+0000", 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20344289300000001 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.224013+0000", 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.075267071000000005 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.224178+0000", 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016516400000000001 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:564 3.cs0 3:35606aaf:::smithi14230322-20:head [setxattr _header (57) in=7b] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.539769+0000", 2024-04-18T23:11:44.069 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.4467238890000003, 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.78455714300000001, 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 564 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539769+0000", 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539769+0000", 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539771+0000", 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2470000000000001e-06 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539777+0000", 2024-04-18T23:11:44.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0730000000000001e-06 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539788+0000", 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0924000000000001e-05 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539795+0000", 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8159999999999994e-06 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.046902+0000", 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50710653999999999 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.047084+0000", 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000181609 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.047936+0000", 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00085221099999999996 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148748+0000", 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100812007 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.324156+0000", 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17540829599999999 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.324327+0000", 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017042000000000001 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.074 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:565 3.cs0 3:35606aaf:::smithi14230322-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.539807+0000", 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.4466863029999999, 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0887034069999999, 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 565 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539807+0000", 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539807+0000", 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539808+0000", 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9299999999999997e-07 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539811+0000", 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5700000000000001e-06 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539822+0000", 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0467e-05 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539829+0000", 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5870000000000004e-06 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148486+0000", 2024-04-18T23:11:44.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60865698899999998 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148648+0000", 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016127099999999999 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.326655+0000", 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17800708000000001 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.326971+0000", 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031639400000000001 2024-04-18T23:11:44.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.628510+0000", 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30153925599999998 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 72/31 ECSubWrite(tid=224, reqid=osd.4.0:222, at_version=0'0, trim_to=0'0, roll_forward_to=72'111))", 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:39.511725+0000", 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.4747686550000001, 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30299252599999998, 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.079 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511725+0000", 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511725+0000", 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511726+0000", 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1999999999999998e-07 2024-04-18T23:11:44.080 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511748+0000", 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2276000000000002e-05 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511772+0000", 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3932999999999999e-05 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.611922+0000", 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100150379 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.081 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.712249+0000", 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100326822 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.712286+0000", 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6962999999999998e-05 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.814658+0000", 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102371988 2024-04-18T23:11:44.082 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.814717+0000", 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9244999999999999e-05 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.083 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:44.085 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:11:44.085 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:44.083+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:44.185 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:44.182+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:44.253 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:44.253 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:44.253 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:44.253 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:44.268 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-18T23:11:44.314 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:44.315+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:44.387 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:44.387+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:44.391 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:44.391 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:44.391 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:44.391 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:44.391 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:44.405 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-18T23:11:44.487 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:44.485+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:44.498 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:44.573 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDPGPush(3.9s4 70/68 [PushOp(3:9e439d70:::smithi14230322-3:head, version: 60'39, data_included: [0~28672], data_size: 28672, omap_header_size: 0, omap_entries_size: 0, attrset_size: 4, recovery_info: ObjectRecoveryInfo(3:9e439d70:::smithi14230322-3:head@60'39, size: 114688, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 114688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:36.947953+0000", 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.623726918, 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101234518, 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:36.947953+0000", 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:36.947953+0000", 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:36.947955+0000", 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6840000000000001e-06 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:36.948012+0000", 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6861999999999999e-05 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:36.948057+0000", 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5809000000000001e-05 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:36.948113+0000", 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.5646000000000002e-05 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.048447+0000", 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10033439800000001 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.049188+0000", 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00074011900000000002 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.577 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s4 70/68 ECSubWrite(tid=142, reqid=client.4568.0:546, at_version=0'0, trim_to=0'0, roll_forward_to=70'56, backfill_or_async_recovery))", 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:37.050622+0000", 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.521057689, 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101633111, 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.050622+0000", 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.050622+0000", 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.050624+0000", 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7400000000000001e-06 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.050651+0000", 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7317e-05 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.050682+0000", 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0831999999999999e-05 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.050708+0000", 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6177e-05 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.150962+0000", 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10025341 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.150980+0000", 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8272999999999999e-05 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.152155+0000", 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011745639999999999 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.152255+0000", 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010079800000000001 2024-04-18T23:11:44.581 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDPGPush(3.6s3 70/68 [PushOp(3:635103ff:::smithi14230322-31:16, version: 69'21, data_included: [0~262144], data_size: 262144, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:635103ff:::smithi14230322-31:16@69'21, size: 1294336, copy_subset: [], clone_subset: {}, snapset: 16=[]:{e: [d, c, b, a, 9, 7], 16: [16, 15, 14, 13, 12, 11]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:37.064490+0000", 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.507190305, 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10165289600000001, 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.064490+0000", 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.064490+0000", 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.064491+0000", 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4670000000000001e-06 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.064911+0000", 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00041958799999999998 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.064956+0000", 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4829000000000001e-05 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.064986+0000", 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0315999999999999e-05 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.165332+0000", 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10034596899999999 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.166143+0000", 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00081072700000000002 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDPGPush(3.9s4 70/68 [PushOp(3:9a6c55c5:::smithi14230322-22:14, version: 65'41, data_included: [0~90112], data_size: 90112, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:9a6c55c5:::smithi14230322-22:14@65'41, size: 360448, copy_subset: [], clone_subset: {}, snapset: 16=[]:{14: [14, 13, 12, 11, f, d, c, 9], 16: [16, 15]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 360448, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-18T23:11:44.585 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:37.258111+0000", 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 7.3135689160000004, 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101615291, 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.258111+0000", 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.258111+0000", 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.258113+0000", 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.86e-06 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.258276+0000", 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016309000000000001 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.258325+0000", 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.9359e-05 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.358497+0000", 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100171573 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.358744+0000", 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00024686800000000002 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.359726+0000", 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00098254099999999997 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDPGPush(3.9s4 71/68 [PushOp(3:9a6c55c5:::smithi14230322-22:16, version: 69'48, data_included: [262144~180224], data_size: 180224, omap_header_size: 0, omap_entries_size: 0, attrset_size: 0, recovery_info: ObjectRecoveryInfo(3:9a6c55c5:::smithi14230322-22:16@69'48, size: 1769472, copy_subset: [], clone_subset: {}, snapset: 16=[]:{14: [14, 13, 12, 11, f, d, c, 9], 16: [16, 15]}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1769472, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(!first, data_recovered_to: 1048576, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:37.574165+0000", 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.9975150519999998, 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101784577, 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.589 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.574165+0000", 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.574165+0000", 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.574166+0000", 2024-04-18T23:11:44.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.172e-06 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.574421+0000", 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000254985 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.574453+0000", 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1664e-05 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.574477+0000", 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4235000000000001e-05 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.674816+0000", 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100339391 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.675950+0000", 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00113313 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDPGPush(3.9s4 71/68 [PushOp(3:9a6c55c5:::smithi14230322-22:head, version: 69'49, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:9a6c55c5:::smithi14230322-22:head@69'49, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:37.771993+0000", 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.7996873789999999, 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101104526, 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.771993+0000", 2024-04-18T23:11:44.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.771993+0000", 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.771995+0000", 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9400000000000001e-06 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.772023+0000", 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7977e-05 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.772086+0000", 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3017999999999999e-05 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.772136+0000", 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.0362000000000003e-05 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.872464+0000", 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100327764 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.873097+0000", 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00063346500000000005 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDPGPush(3.6s3 71/68 [PushOp(3:635103ff:::smithi14230322-31:head, version: 69'22, data_included: [], data_size: 0, omap_header_size: 0, omap_entries_size: 0, attrset_size: 3, recovery_info: ObjectRecoveryInfo(3:635103ff:::smithi14230322-31:head@69'22, size: 0, copy_subset: [], clone_subset: {}, snapset: 0=[]:{}, object_exist: true), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 0, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: true, error: false))])", 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:37.785366+0000", 2024-04-18T23:11:44.596 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.7863141890000005, 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101094082, 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.785366+0000", 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.785366+0000", 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.785366+0000", 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0400000000000002e-07 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.785385+0000", 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8663000000000001e-05 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.785432+0000", 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7156e-05 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.785477+0000", 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.4808999999999997e-05 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.885816+0000", 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10033953599999999 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:37.886460+0000", 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00064351399999999998 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.8s5 71/70 ECSubWrite(tid=35, reqid=osd.5.0:35, at_version=71'26, trim_to=0'0, roll_forward_to=71'22))", 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.188325+0000", 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 6.3833545039999997, 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10250044, 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.600 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.188325+0000", 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.188325+0000", 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.188326+0000", 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.7000000000000005e-07 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.188363+0000", 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7212999999999998e-05 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.188378+0000", 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4250000000000001e-05 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.288543+0000", 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10016528500000001 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.288762+0000", 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021949900000000001 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.288793+0000", 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0533000000000001e-05 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.290731+0000", 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019377280000000001 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.290826+0000", 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5361999999999999e-05 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 72/67 ECSubWrite(tid=205, reqid=osd.13.0:205, at_version=72'35, trim_to=0'0, roll_forward_to=68'33))", 2024-04-18T23:11:44.604 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.614968+0000", 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.9567116950000001, 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102025402, 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.614968+0000", 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.614968+0000", 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.614970+0000", 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8840000000000001e-06 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.615054+0000", 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4264999999999994e-05 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.615083+0000", 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8691999999999999e-05 2024-04-18T23:11:44.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.715177+0000", 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100093766 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.715334+0000", 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015678400000000001 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.715362+0000", 2024-04-18T23:11:44.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8496e-05 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.716899+0000", 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00153688 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.716994+0000", 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4635000000000004e-05 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.8s5 72/70 ECSubWrite(tid=38, reqid=osd.5.0:37, at_version=0'0, trim_to=0'0, roll_forward_to=71'28))", 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.714243+0000", 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.857436989, 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102543044, 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.714243+0000", 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.714243+0000", 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.714244+0000", 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6000000000000004e-07 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.714268+0000", 2024-04-18T23:11:44.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4012e-05 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.714309+0000", 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1387000000000002e-05 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.814425+0000", 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100115652 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.814617+0000", 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019224699999999999 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.814658+0000", 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0918999999999999e-05 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.816655+0000", 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019975930000000002 2024-04-18T23:11:44.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.816786+0000", 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000130674 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.8s5 72/70 ECSubWrite(tid=40, reqid=osd.5.0:39, at_version=0'0, trim_to=0'0, roll_forward_to=72'30))", 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:39.021749+0000", 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.5499311220000003, 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101381586, 2024-04-18T23:11:44.613 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.021749+0000", 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.021749+0000", 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.021750+0000", 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6330000000000001e-06 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.021774+0000", 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3023e-05 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.021798+0000", 2024-04-18T23:11:44.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4547000000000001e-05 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.021818+0000", 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0035e-05 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.021924+0000", 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010592899999999999 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.021943+0000", 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8606000000000001e-05 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.123043+0000", 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101100695 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.123130+0000", 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7118000000000002e-05 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.617 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 73/67 ECSubWrite(tid=207, reqid=client.4568.0:572, at_version=73'37, trim_to=0'0, roll_forward_to=72'35))", 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:39.524763+0000", 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.0469169889999996, 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10546235399999999, 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.524763+0000", 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.524763+0000", 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.524764+0000", 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.469e-06 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.525383+0000", 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00061890299999999997 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.525423+0000", 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.9779e-05 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.525447+0000", 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4176e-05 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.626086+0000", 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100639179 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.626115+0000", 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8459e-05 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.630156+0000", 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040409110000000003 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.630225+0000", 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9478000000000001e-05 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 73/67 ECSubWrite(tid=211, reqid=client.4568.0:576, at_version=73'41, trim_to=0'0, roll_forward_to=72'35))", 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:39.546190+0000", 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.0254899709999998, 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101472834, 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.622 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.546190+0000", 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.546190+0000", 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.546191+0000", 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2669999999999999e-06 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.546289+0000", 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7424999999999996e-05 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.546318+0000", 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9609999999999999e-05 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.646453+0000", 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10013480800000001 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.646609+0000", 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015570699999999999 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.646637+0000", 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7708999999999998e-05 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.647604+0000", 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00096714900000000005 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.647663+0000", 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9159000000000001e-05 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.8s5 73/70 ECSubWrite(tid=41, reqid=osd.5.0:41, at_version=73'32, trim_to=0'0, roll_forward_to=72'30))", 2024-04-18T23:11:44.626 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:39.611913+0000", 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.9597669839999998, 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103174899, 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.611913+0000", 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.611913+0000", 2024-04-18T23:11:44.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.611913+0000", 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4999999999999998e-07 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.611970+0000", 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6511999999999997e-05 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.612005+0000", 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5423000000000003e-05 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.712157+0000", 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100151432 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.712355+0000", 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019862699999999999 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.712389+0000", 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3516000000000002e-05 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.715020+0000", 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00263152 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.715088+0000", 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.7519000000000002e-05 2024-04-18T23:11:44.630 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.6s3 73/72 ECSubWrite(tid=37, reqid=osd.0.0:37, at_version=73'24, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:39.721444+0000", 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.8502358020000003, 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102231001, 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.721444+0000", 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.721444+0000", 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.721446+0000", 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.624e-06 2024-04-18T23:11:44.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.721530+0000", 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4312000000000002e-05 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.721566+0000", 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5932000000000001e-05 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.821702+0000", 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10013559 2024-04-18T23:11:44.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.821893+0000", 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019099300000000001 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.821918+0000", 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.525e-05 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.823580+0000", 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0016624280000000001 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.823675+0000", 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.4871999999999996e-05 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.8s5 73/70 ECSubWrite(tid=42, reqid=osd.5.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=73'32))", 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:39.821839+0000", 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 4.7498412200000004, 2024-04-18T23:11:44.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101384139, 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.821839+0000", 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.821839+0000", 2024-04-18T23:11:44.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.821841+0000", 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0439999999999998e-06 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.821957+0000", 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000116018 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.821990+0000", 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2835e-05 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.922113+0000", 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100123183 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.922287+0000", 2024-04-18T23:11:44.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017366300000000001 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.922317+0000", 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0190000000000001e-05 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.923128+0000", 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00081108099999999995 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.923223+0000", 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5124999999999995e-05 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.9s4 75/72 ECSubWrite(tid=171, reqid=osd.14.0:170, at_version=0'0, trim_to=0'0, roll_forward_to=75'76))", 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:42.524330+0000", 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 2.0473495920000002, 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10120284, 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.524330+0000", 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.524330+0000", 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.524333+0000", 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2199999999999999e-06 2024-04-18T23:11:44.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.524364+0000", 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1389000000000001e-05 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.524412+0000", 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7979000000000003e-05 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.524439+0000", 2024-04-18T23:11:44.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7455999999999999e-05 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.624672+0000", 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100232691 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.624691+0000", 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8597000000000002e-05 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.625406+0000", 2024-04-18T23:11:44.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00071562800000000005 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:42.625533+0000", 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012688 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:633 3.5s0 3:a09a6894:::smithi14230322-25:head [create,setxattr UrPUWbaU]cNfinVtoVUNsYX\\ahn\\oSdRW\\N[RcfO (807) in=40b,setxattr ZQoeVnpeaooblfNZklrg\\hmphZ^pTVh`eRXheade (1225) in=40b,setxattr kcjR][ah`rcmsWjUVlUeaofblgmpleUjjs^]ai\\t (7928) in=40b,setxattr `U^gRRs]UWRVngUaNeN^`p`aVl\\cn_a\\S`iTpOVo (1156) in=40b,setxattr aOYbW^`[gQfVcTQrq[NYeqh\\Q_bjiaWOefngkiNR (1331) in=40b,setxattr igNjm (735) in=5b,setxattr aQYs^c[hafOm^\\g[^R\\^QOd^eiS\\ghimncRcb_pi (15856) in=40b,setxattr dZUpqXmQYdS]obbkNRYdSsdUpenRkjiSSfUmqRTd (9612) in=40b,setxattr lX^flfV^[d^WmlW_UT^kV`s^ahfcNQsT`de`^spi (1612) in=40b,setxattr Ua`UXimY^dWVoQaji\\cZpP^er^hlbYQZs`USR`oh (10033) in=40b,setxattr XhOccedqUYhrPkk[^dVtangb_r^]alTl`ogaOPag (840) in=40b,setxattr YqbXbb\\oPeeR]rqcNaSsS`lkYoffXPcpaqhrb^te (418) in=40b,setxattr jdOeY]PQnfd[iZ^ph\\ijd[WRRnQ^phTpRROYeXss (1611) in=40b,setxattr ie\\ahhoaskiWrasYNh\\mUkqRso\\Olj\\c^P_\\bPp\\ (15577) in=40b,setxattr Ptb\\gnjjhTnd`OQag[dXTYa]t_id[PRpjt_ZfVY] (1541) in=40b,setxattr sSR]a[NPVaUQhPkla[ZVX`lTdim`fgjgeRtm`tt^ (841) in=40b,setxattr _cf[gnNmUkknhenPpY^\\lefOR_n\\mcb`iqOOPUiW (11296) in=40b,setxattr d[TVh[TTre^P`mnfob^WP[dckRkQZafjR[PaW^Zn (8912) in=40b,setxattr X[gWOjQtntoVfbZV[UObOVPoj`qib[[Zt]trgVjh (1542) in=40b,setxattr _qa\\VSbX\\\\b`tO_lSRnsf`PVNfmPkWrf[P\\ZTej[ (453) in=40b] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:43.860879+0000", 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.71080140199999997, 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.27188362799999999, 2024-04-18T23:11:44.644 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 633 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.860879+0000", 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.860879+0000", 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.860882+0000", 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5729999999999999e-06 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.861048+0000", 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016603399999999999 2024-04-18T23:11:44.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.861075+0000", 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6982999999999999e-05 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.861111+0000", 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5448999999999999e-05 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.861277+0000", 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000166467 2024-04-18T23:11:44.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.861540+0000", 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026288599999999999 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.864071+0000", 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025311750000000001 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:43.967370+0000", 2024-04-18T23:11:44.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10329856699999999 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.132589+0000", 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.16521914800000001 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.132762+0000", 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000173346 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 77/72 ECSubWriteReply(tid=15, last_complete=77'60, committed=1, applied=1))", 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:44.134615+0000", 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.43706499500000001, 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100324339, 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134615+0000", 2024-04-18T23:11:44.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134615+0000", 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134616+0000", 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4729999999999999e-06 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134625+0000", 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8130000000000002e-06 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134650+0000", 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4437e-05 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134665+0000", 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5065999999999999e-05 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.234899+0000", 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100234024 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.234939+0000", 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0525999999999998e-05 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 77/72 ECSubWriteReply(tid=15, last_complete=77'60, committed=1, applied=1))", 2024-04-18T23:11:44.653 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:44.134751+0000", 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.43692913100000003, 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100355264, 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134751+0000", 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:44.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134751+0000", 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134753+0000", 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.807e-06 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134764+0000", 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.099e-05 2024-04-18T23:11:44.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134795+0000", 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1795000000000003e-05 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.134809+0000", 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.342e-05 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:44.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.235077+0000", 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10026861400000001 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.235106+0000", 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8637999999999999e-05 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:44.657 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:44.658 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T23:11:44.659 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:44.657+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:44.660 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.7 to 1 (10000) 2024-04-18T23:11:44.661 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-04-18T23:11:44.661 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:11:44.662 INFO:tasks.rados.rados.0.smithi142.stdout:264: done (8 left) 2024-04-18T23:11:44.662 INFO:tasks.rados.rados.0.smithi142.stdout:265: read oid 46 snap -1 2024-04-18T23:11:44.662 INFO:tasks.rados.rados.0.smithi142.stdout:265: expect (ObjNum 6 snap 0 seq_num 6) 2024-04-18T23:11:44.662 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 23 current snap is 25 2024-04-18T23:11:44.662 INFO:tasks.rados.rados.0.smithi142.stdout:266: seq_num 105 ranges {2441216=606208,3047424=688128,3735552=475136} 2024-04-18T23:11:44.662 INFO:tasks.rados.rados.0.smithi142.stdout:266: writing smithi14230322-23 from 2441216 to 3047424 tid 1 2024-04-18T23:11:44.664 INFO:tasks.rados.rados.0.smithi142.stdout:266: writing smithi14230322-23 from 3047424 to 3735552 tid 2 2024-04-18T23:11:44.670 INFO:tasks.rados.rados.0.smithi142.stdout:266: writing smithi14230322-23 from 3735552 to 4210688 tid 3 2024-04-18T23:11:44.670 INFO:tasks.rados.rados.0.smithi142.stdout:267: read oid 5 snap -1 2024-04-18T23:11:44.670 INFO:tasks.rados.rados.0.smithi142.stdout:267: expect deleted 2024-04-18T23:11:44.671 INFO:tasks.rados.rados.0.smithi142.stdout:268: snap_remove snap 24 2024-04-18T23:11:44.738 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:44.739 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:44.759 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:44.759+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:44.832 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:44.832 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:44.845 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:44.846 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:44.852 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:11:44.860 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:44.857+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:44.871 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:44.871 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:44.892 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:44.892 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:44.960 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:44.957+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:44.972 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:44.972 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:44.986 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:11:45.037 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:45.038 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:45.058 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:45.058 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:45.061 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:45.058+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:45.078 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:45.079 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:45.100 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:45.100 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:45.144 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:45.144 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:45.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:11:45.163 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:45.163+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:45.241 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:45.241 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:45.264 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:45.262+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:45.266 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:45.267 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:45.288 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:45.288 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:45.308 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:45.309 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:45.323 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:45.323 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:45.327 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:45.327 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:45.340 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:11:45.365 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:45.361+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:45.400 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:45.400 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:45.420 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:45.421 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:45.443 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:45.443 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:45.466 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:45.463+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:45.507 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:45.507 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:45.520 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:11:45.566 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:45.561+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:45.610 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.8 to 1 (10000) 2024-04-18T23:11:45.610 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:268: done (11 left) 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:269: copy_from oid 48 from oid 30 current snap is 25 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:253: finishing write tid 1 to smithi14230322-45 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:259: finishing rollback tid 0 to smithi14230322-14 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:259: finishing rollback tid 1 to smithi14230322-14 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 25 v 60 (ObjNum 56 snap 4 seq_num 56) dirty exists 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 15 v 34 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:253: finishing write tid 2 to smithi14230322-45 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:253: finishing write tid 3 to smithi14230322-45 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:253: finishing write tid 4 to smithi14230322-45 2024-04-18T23:11:45.613 INFO:tasks.rados.rados.0.smithi142.stdout:252: expect (ObjNum 46 snap 0 seq_num 46) 2024-04-18T23:11:45.639 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-04-18T23:11:45.667 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:45.665+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:45.669 INFO:tasks.rados.rados.0.smithi142.stdout:259: finishing rollback tid 2 to smithi14230322-14 2024-04-18T23:11:45.669 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 14 v 130 (ObjNum 13 snap 0 seq_num 13) dirty exists 2024-04-18T23:11:45.669 INFO:tasks.rados.rados.0.smithi142.stdout:254: expect (ObjNum 64 snap 7 seq_num 64) 2024-04-18T23:11:45.683 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:45.683 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:45.697 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:11:45.749 INFO:tasks.rados.rados.0.smithi142.stdout:253: finishing write tid 5 to smithi14230322-45 2024-04-18T23:11:45.749 INFO:tasks.rados.rados.0.smithi142.stdout:253: oid 45 updating version 0 to 84 2024-04-18T23:11:45.749 INFO:tasks.rados.rados.0.smithi142.stdout:253: oid 45 updating version 84 to 85 2024-04-18T23:11:45.749 INFO:tasks.rados.rados.0.smithi142.stdout:253: oid 45 updating version 85 to 87 2024-04-18T23:11:45.749 INFO:tasks.rados.rados.0.smithi142.stdout:253: oid 45 version 87 is already newer than 86 2024-04-18T23:11:45.749 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 45 v 87 (ObjNum 102 snap 25 seq_num 102) dirty exists 2024-04-18T23:11:45.749 INFO:tasks.rados.rados.0.smithi142.stdout:253: left oid 45 (ObjNum 102 snap 25 seq_num 102) 2024-04-18T23:11:45.749 INFO:tasks.rados.rados.0.smithi142.stdout:261: finishing write tid 1 to smithi14230322-19 2024-04-18T23:11:45.749 INFO:tasks.rados.rados.0.smithi142.stdout:256: expect (ObjNum 98 snap 21 seq_num 98) 2024-04-18T23:11:45.768 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:45.767+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:45.799 INFO:tasks.rados.rados.0.smithi142.stdout:261: finishing write tid 2 to smithi14230322-19 2024-04-18T23:11:45.799 INFO:tasks.rados.rados.0.smithi142.stdout:261: finishing write tid 3 to smithi14230322-19 2024-04-18T23:11:45.799 INFO:tasks.rados.rados.0.smithi142.stdout:261: finishing write tid 4 to smithi14230322-19 2024-04-18T23:11:45.799 INFO:tasks.rados.rados.0.smithi142.stdout:261: finishing write tid 5 to smithi14230322-19 2024-04-18T23:11:45.799 INFO:tasks.rados.rados.0.smithi142.stdout:261: oid 19 updating version 0 to 43 2024-04-18T23:11:45.799 INFO:tasks.rados.rados.0.smithi142.stdout:261: oid 19 updating version 43 to 45 2024-04-18T23:11:45.799 INFO:tasks.rados.rados.0.smithi142.stdout:261: oid 19 version 45 is already newer than 42 2024-04-18T23:11:45.799 INFO:tasks.rados.rados.0.smithi142.stdout:261: oid 19 version 45 is already newer than 44 2024-04-18T23:11:45.799 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 19 v 45 (ObjNum 104 snap 25 seq_num 104) dirty exists 2024-04-18T23:11:45.800 INFO:tasks.rados.rados.0.smithi142.stdout:261: left oid 19 (ObjNum 104 snap 25 seq_num 104) 2024-04-18T23:11:45.800 INFO:tasks.rados.rados.0.smithi142.stdout:265: expect (ObjNum 6 snap 0 seq_num 6) 2024-04-18T23:11:45.802 INFO:tasks.rados.rados.0.smithi142.stdout:266: finishing write tid 1 to smithi14230322-23 2024-04-18T23:11:45.802 INFO:tasks.rados.rados.0.smithi142.stdout:266: finishing write tid 2 to smithi14230322-23 2024-04-18T23:11:45.802 INFO:tasks.rados.rados.0.smithi142.stdout:266: finishing write tid 3 to smithi14230322-23 2024-04-18T23:11:45.802 INFO:tasks.rados.rados.0.smithi142.stdout:266: finishing write tid 4 to smithi14230322-23 2024-04-18T23:11:45.802 INFO:tasks.rados.rados.0.smithi142.stdout:252: done (11 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:253: done (10 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:254: done (9 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:256: done (8 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:259: done (7 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:261: done (6 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:262: done (5 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:263: done (4 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:265: done (3 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:267: done (2 left) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:270: read oid 43 snap -1 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:270: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-18T23:11:45.803 INFO:tasks.rados.rados.0.smithi142.stdout:271: snap_remove snap 10 2024-04-18T23:11:45.854 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:45.854 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:45.867 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:11:45.869 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:45.867+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:45.970 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:45.967+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:45.973 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:45.974 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:45.989 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:11:46.070 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:46.071+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:46.095 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:46.095 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:46.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:11:46.172 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:46.171+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:46.224 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:46.224 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:46.238 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:11:46.284 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:46.282+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:46.356 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:46.356 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:46.370 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:11:46.373 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:46.371+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:46.474 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:46.470+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:46.529 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:46.529 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:46.542 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:11:46.575 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:46.575+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:271: done (3 left) 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:272: read oid 40 snap -1 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:272: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:273: delete oid 46 current snap is 25 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:266: finishing write tid 5 to smithi14230322-23 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:266: oid 23 updating version 0 to 93 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:266: oid 23 updating version 93 to 94 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:266: oid 23 updating version 94 to 95 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:266: oid 23 updating version 95 to 96 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 23 v 96 (ObjNum 105 snap 25 seq_num 105) dirty exists 2024-04-18T23:11:46.622 INFO:tasks.rados.rados.0.smithi142.stdout:266: left oid 23 (ObjNum 105 snap 25 seq_num 105) 2024-04-18T23:11:46.623 INFO:tasks.rados.rados.0.smithi142.stdout:270: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-18T23:11:46.623 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.9 to 1 (10000) 2024-04-18T23:11:46.645 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-04-18T23:11:46.675 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:46.675+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:46.678 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:46.679 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:46.693 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:11:46.734 INFO:tasks.rados.rados.0.smithi142.stdout:269: finishing copy_from racing read to smithi14230322-48 2024-04-18T23:11:46.734 INFO:tasks.rados.rados.0.smithi142.stdout:269: finishing copy_from to smithi14230322-48 2024-04-18T23:11:46.734 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 48 v 4 (ObjNum 29 snap 0 seq_num 29) dirty exists 2024-04-18T23:11:46.775 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:46.770+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:46.837 INFO:tasks.rados.rados.0.smithi142.stdout:266: done (4 left) 2024-04-18T23:11:46.837 INFO:tasks.rados.rados.0.smithi142.stdout:269: done (3 left) 2024-04-18T23:11:46.838 INFO:tasks.rados.rados.0.smithi142.stdout:270: done (2 left) 2024-04-18T23:11:46.838 INFO:tasks.rados.rados.0.smithi142.stdout:273: done (1 left) 2024-04-18T23:11:46.838 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 10 current snap is 25 2024-04-18T23:11:46.838 INFO:tasks.rados.rados.0.smithi142.stdout:274: seq_num 107 ranges {786432=638976,1425408=458752,1884160=704512} 2024-04-18T23:11:46.859 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:46.859 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:46.861 INFO:tasks.rados.rados.0.smithi142.stdout:274: writing smithi14230322-10 from 786432 to 1425408 tid 1 2024-04-18T23:11:46.869 INFO:tasks.rados.rados.0.smithi142.stdout:274: writing smithi14230322-10 from 1425408 to 1884160 tid 2 2024-04-18T23:11:46.873 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:11:46.876 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:46.877+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:46.880 INFO:tasks.rados.rados.0.smithi142.stdout:274: writing smithi14230322-10 from 1884160 to 2588672 tid 3 2024-04-18T23:11:46.881 INFO:tasks.rados.rados.0.smithi142.stdout:275: rmattr oid 7 current snap is 25 2024-04-18T23:11:46.881 INFO:tasks.rados.rados.0.smithi142.stdout:275: done (2 left) 2024-04-18T23:11:46.881 INFO:tasks.rados.rados.0.smithi142.stdout:276: snap_remove snap 15 2024-04-18T23:11:46.978 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:46.974+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:46.998 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:46.998 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:47.012 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:11:47.079 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:47.074+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:47.132 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:47.132 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:47.147 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:11:47.191 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:47.191+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:47.269 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:47.269 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:47.280 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:47.279+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:47.283 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:11:47.381 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:47.381+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:47.451 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:47.451 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:47.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:47.466 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:11:47.481 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:47.479+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:47.582 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:47.583+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:47.588 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:47.588 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:47.603 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:11:47.616 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:47.616 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:47.627 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.10 to 1 (10000) 2024-04-18T23:11:47.630 INFO:tasks.rados.rados.0.smithi142.stdout:276: done (2 left) 2024-04-18T23:11:47.631 INFO:tasks.rados.rados.0.smithi142.stdout:277: snap_remove snap 22 2024-04-18T23:11:47.632 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T23:11:47.637 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:47.637 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:47.651 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-04-18T23:11:47.661 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:47.662 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:47.704 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:47.702+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:47.724 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:47.724 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:47.740 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:11:47.769 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:47.769 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:47.784 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:47.783+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:47.863 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:47.863 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:47.877 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:11:47.885 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:47.883+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:47.970 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:47.970 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:47.985 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:47.982+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:47.987 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:47.987 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:48.003 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:48.003 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:48.008 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:48.008 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:48.018 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:11:48.033 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:48.034 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:48.055 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:48.056 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:48.074 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:48.074 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:48.086 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:48.087+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:48.144 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:48.145 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:48.159 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:11:48.170 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:48.170 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:48.187 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:48.182+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:48.197 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:48.198 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:48.225 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:48.226 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:48.254 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:48.254 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:48.279 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:48.280 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:48.288 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:48.287+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.736456621999999, 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62069844100000005, 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.333 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 114 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594565+0000", 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594570+0000", 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.186e-06 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594582+0000", 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1763000000000001e-05 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594590+0000", 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5290000000000003e-06 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010572+0000", 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.415982252 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010727+0000", 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015558400000000001 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010971+0000", 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000243339 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015929+0000", 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049583639999999998 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215108+0000", 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19917934400000001 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215199+0000", 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0556000000000005e-05 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215263+0000", 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3874000000000005e-05 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:48.338 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.736420295000002, 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62089894099999998, 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 115 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594746+0000", 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014446700000000001 2024-04-18T23:11:48.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594761+0000", 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4792e-05 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594769+0000", 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6160000000000002e-06 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013742+0000", 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.418972438 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013974+0000", 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023265699999999999 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.014328+0000", 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035345 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:48.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018069+0000", 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037411079999999999 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215346+0000", 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19727757900000001 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215435+0000", 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8245999999999995e-05 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215500+0000", 2024-04-18T23:11:48.343 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4968000000000005e-05 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.736237043999999, 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.72133309899999998, 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.344 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 116 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.345 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594785+0000", 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594790+0000", 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4229999999999996e-06 2024-04-18T23:11:48.346 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594803+0000", 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2196e-05 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594810+0000", 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6190000000000002e-06 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016062+0000", 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42125235 2024-04-18T23:11:48.347 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016215+0000", 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000152051 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016420+0000", 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000205916 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018082+0000", 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001661718 2024-04-18T23:11:48.348 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.315968+0000", 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29788616600000001 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316051+0000", 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2897000000000003e-05 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316117+0000", 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5866999999999998e-05 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.349 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.736199411000001, 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.721588585, 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 117 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.350 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:48.351 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594829+0000", 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7830000000000001e-06 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594841+0000", 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1505000000000001e-05 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.694940+0000", 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100099585 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.352 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016967+0000", 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32202692799999999 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017224+0000", 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000256548 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017583+0000", 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000359689 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.353 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.020139+0000", 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025556649999999999 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316256+0000", 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29611688800000002 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316345+0000", 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9285999999999999e-05 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.354 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316410+0000", 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5058e-05 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:162 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.196938914999997, 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50946670199999999, 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.355 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 162 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.356 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134082+0000", 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134083+0000", 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7000000000000003e-07 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134088+0000", 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2719999999999997e-06 2024-04-18T23:11:48.357 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134097+0000", 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.1400000000000006e-06 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134104+0000", 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8589999999999999e-06 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140241+0000", 2024-04-18T23:11:48.358 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0061371919999999996 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140319+0000", 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.7768000000000005e-05 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140446+0000", 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000126637 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:48.359 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243778+0000", 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.103331863 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643391+0000", 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.39961375599999999 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643476+0000", 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4274e-05 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.360 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643549+0000", 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3071000000000006e-05 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:163 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.196905755000003, 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50963481200000005, 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:48.361 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 163 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134115+0000", 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.362 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134116+0000", 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134123+0000", 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4930000000000003e-06 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134131+0000", 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9490000000000002e-06 2024-04-18T23:11:48.363 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134138+0000", 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.8399999999999997e-06 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140848+0000", 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0067093389999999999 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.140969+0000", 2024-04-18T23:11:48.364 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000121787 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141164+0000", 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019482700000000001 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243782+0000", 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10261785900000001 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643626+0000", 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.399843858 2024-04-18T23:11:48.365 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643699+0000", 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2657999999999995e-05 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643750+0000", 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1481999999999999e-05 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.366 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:164 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.196871664, 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50988570200000005, 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 164 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:48.367 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134149+0000", 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134150+0000", 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4000000000000001e-07 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134332+0000", 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018177199999999999 2024-04-18T23:11:48.368 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134351+0000", 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9321999999999999e-05 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134368+0000", 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6994999999999999e-05 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141740+0000", 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0073716420000000003 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.369 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141823+0000", 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3042999999999998e-05 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.141951+0000", 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000127976 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.243787+0000", 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101836063 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643874+0000", 2024-04-18T23:11:48.370 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40008676199999998 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.643964+0000", 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0362e-05 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.644035+0000", 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.1124999999999994e-05 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.371 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:165 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e28)", 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 58.196612428000002, 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.54739116099999996, 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 165 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.372 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134409+0000", 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134410+0000", 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5770000000000001e-06 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134420+0000", 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0175e-05 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.373 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134439+0000", 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8125e-05 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.134453+0000", 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4956e-05 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242648+0000", 2024-04-18T23:11:48.374 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.108194819 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242783+0000", 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013431599999999999 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.242992+0000", 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020903 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.244280+0000", 2024-04-18T23:11:48.375 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012883669999999999 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681587+0000", 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.43730722300000002 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681676+0000", 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8776000000000005e-05 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.376 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:50.681800+0000", 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012379700000000001 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:11 3.2s0 3:42341748:::smithi14230322-4:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.812722377999997, 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50900182400000005, 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.377 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 11 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518299+0000", 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.378 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518300+0000", 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.823e-06 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518324+0000", 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3238999999999999e-05 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518345+0000", 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0828000000000001e-05 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.379 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.518359+0000", 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4912000000000001e-05 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521623+0000", 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003263808 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.522045+0000", 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00042140500000000002 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.380 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.523231+0000", 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011861840000000001 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530352+0000", 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071208340000000004 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.027150+0000", 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49679807100000001 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.381 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.027300+0000", 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015071999999999999 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:25 3.2s0 3:4250b374:::smithi14230322-9:head [append 0~147456,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.809142477999998, 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.51011498, 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.382 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 25 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521879+0000", 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.521881+0000", 2024-04-18T23:11:48.383 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4430000000000002e-06 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524055+0000", 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021740499999999999 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524086+0000", 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1414000000000001e-05 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524107+0000", 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0716e-05 2024-04-18T23:11:48.384 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524323+0000", 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000215675 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524696+0000", 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00037315500000000001 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526737+0000", 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020405380000000002 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.385 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.530355+0000", 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0036182720000000001 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.031805+0000", 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.50145000299999998 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.031994+0000", 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018871400000000001 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.386 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:26 3.2s0 3:4250b374:::smithi14230322-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.806813384999998, 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60852922899999995, 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 26 2024-04-18T23:11:48.387 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524210+0000", 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.388 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524220+0000", 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0056e-05 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524242+0000", 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2212e-05 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525734+0000", 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001491783 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528099+0000", 2024-04-18T23:11:48.389 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023654380000000001 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528318+0000", 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000219065 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.529513+0000", 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011952530000000001 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.390 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636414+0000", 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106900906 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132457+0000", 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49604301899999997 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132737+0000", 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027950099999999998 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.391 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:31 3.2s0 3:42c34e84:::smithi14230322-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.804698961, 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60793644899999999, 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 31 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.392 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526324+0000", 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9400000000000001e-06 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.393 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526836+0000", 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051192200000000005 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526869+0000", 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2978000000000002e-05 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526890+0000", 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481999999999999e-05 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.394 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531167+0000", 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042762490000000002 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531508+0000", 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034109400000000001 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.533620+0000", 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021122279999999998 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.395 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636418+0000", 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102798509 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134075+0000", 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.497656453 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134259+0000", 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000183594 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.396 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:32 3.2s0 3:42c34e84:::smithi14230322-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 51.804096051000002, 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60874687999999999, 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 32 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.397 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526927+0000", 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0389999999999999e-06 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.398 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526936+0000", 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2599999999999994e-06 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0302e-05 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526973+0000", 2024-04-18T23:11:48.399 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6359e-05 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634473+0000", 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.107499583 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634639+0000", 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016687200000000001 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.635580+0000", 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094034999999999995 2024-04-18T23:11:48.400 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636830+0000", 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012506220000000001 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135509+0000", 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49867835399999999 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.401 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135672+0000", 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000163139 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:618 3.2s0 3:42c34e84:::smithi14230322-11:head [read 0~1835008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.708777134, 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.63639373399999999, 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.402 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 618 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.403 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622245+0000", 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622253+0000", 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4889999999999998e-06 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622276+0000", 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2938999999999998e-05 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.404 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722418+0000", 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100141811 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822632+0000", 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100214391 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822645+0000", 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2863e-05 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.405 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.841571+0000", 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018926235 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.841706+0000", 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000134528 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.258638+0000", 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41693209799999997 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.406 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:619 3.2s0 3:42c34e84:::smithi14230322-11:head [read 0~1835008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.6085615740000003, 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.94106379600000001, 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 619 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.407 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722462+0000", 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.12e-06 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722475+0000", 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3475999999999999e-05 2024-04-18T23:11:48.408 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722500+0000", 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4817999999999999e-05 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722517+0000", 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7452000000000002e-05 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822716+0000", 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100198302 2024-04-18T23:11:48.409 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822729+0000", 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3036e-05 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.842152+0000", 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019423458000000001 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.842286+0000", 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013423100000000001 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.410 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.663523+0000", 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.82123690299999996 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:628 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~589824] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4919061830000002, 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.82267251500000005, 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.411 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 628 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.412 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839118+0000", 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6759999999999999e-06 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840018+0000", 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090027100000000001 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840051+0000", 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3426999999999997e-05 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.413 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840069+0000", 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7869000000000001e-05 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.949596+0000", 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10952640800000001 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.949748+0000", 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001526 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.414 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.951307+0000", 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001558633 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.052856+0000", 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101548762 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.661580+0000", 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.608724088 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.415 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.661787+0000", 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020778100000000001 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:629 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~540672] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.484276682, 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91730556200000002, 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.416 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 629 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.417 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846746+0000", 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5799999999999999e-06 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847461+0000", 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071526900000000004 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847485+0000", 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3509000000000001e-05 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847497+0000", 2024-04-18T23:11:48.418 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2675e-05 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.047393+0000", 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19989558700000001 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.047586+0000", 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000192568 2024-04-18T23:11:48.419 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.049081+0000", 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001495225 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.052942+0000", 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038612070000000002 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.763898+0000", 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.71095591800000002 2024-04-18T23:11:48.420 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764050+0000", 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015202400000000001 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:630 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~622592] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4754146969999997, 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.908871761, 2024-04-18T23:11:48.421 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 630 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.422 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855609+0000", 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.17e-06 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856431+0000", 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082242899999999996 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.423 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856463+0000", 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2320999999999998e-05 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856479+0000", 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5396e-05 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.050092+0000", 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19361344699999999 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.424 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.050246+0000", 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000153497 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.051908+0000", 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001662562 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.154762+0000", 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102854035 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.425 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764322+0000", 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60956000700000001 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764478+0000", 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000155897 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:631 3.2s0 3:4b92d38e:::smithi14230322-19:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.474515029, 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90835886899999996, 2024-04-18T23:11:48.426 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 631 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.427 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856507+0000", 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1000000000000001e-06 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856512+0000", 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2290000000000002e-06 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856534+0000", 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1356e-05 2024-04-18T23:11:48.428 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856543+0000", 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8060000000000001e-06 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.053052+0000", 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19650978299999999 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.053209+0000", 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015716999999999999 2024-04-18T23:11:48.429 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054047+0000", 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000837925 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.154833+0000", 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100785849 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764707+0000", 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60987347400000003 2024-04-18T23:11:48.430 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764865+0000", 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015817699999999999 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:632 3.2s0 3:4b92d38e:::smithi14230322-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4744655260000004, 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.212644217, 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:48.431 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 632 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.432 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0699999999999996e-07 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856560+0000", 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4230000000000001e-06 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856573+0000", 2024-04-18T23:11:48.433 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3009e-05 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856581+0000", 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0669999999999992e-06 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054652+0000", 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198071467 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054999+0000", 2024-04-18T23:11:48.434 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034629799999999998 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.766030+0000", 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.71103122399999996 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.766186+0000", 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015654300000000001 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:45.069200+0000", 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30301327900000002 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.435 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:48.436 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:48.436 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:48.437 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:48.438 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:11:48.439 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:48.439 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:48.434+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:48.508 INFO:tasks.rados.rados.0.smithi142.stdout:277: done (2 left) 2024-04-18T23:11:48.509 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 9 current snap is 25 2024-04-18T23:11:48.509 INFO:tasks.rados.rados.0.smithi142.stdout:278: seq_num 108 ranges {2342912=655360,2998272=606208,3604480=475136,4079616=81920} 2024-04-18T23:11:48.511 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-04-18T23:11:48.511 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.11 to 1 (10000) 2024-04-18T23:11:48.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-04-18T23:11:48.538 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:48.537+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:48.557 INFO:tasks.rados.rados.0.smithi142.stdout:278: writing smithi14230322-9 from 2342912 to 2998272 tid 1 2024-04-18T23:11:48.566 INFO:tasks.rados.rados.0.smithi142.stdout:278: writing smithi14230322-9 from 2998272 to 3604480 tid 2 2024-04-18T23:11:48.571 INFO:tasks.rados.rados.0.smithi142.stdout:278: writing smithi14230322-9 from 3604480 to 4079616 tid 3 2024-04-18T23:11:48.572 INFO:tasks.rados.rados.0.smithi142.stdout:278: writing smithi14230322-9 from 4079616 to 4161536 tid 4 2024-04-18T23:11:48.572 INFO:tasks.rados.rados.0.smithi142.stdout:279: read oid 30 snap -1 2024-04-18T23:11:48.572 INFO:tasks.rados.rados.0.smithi142.stdout:279: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-18T23:11:48.573 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 42 current snap is 25 2024-04-18T23:11:48.573 INFO:tasks.rados.rados.0.smithi142.stdout:280: seq_num 109 ranges {704512=671744,1376256=753664,2129920=409600} 2024-04-18T23:11:48.573 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:48.573 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:48.573 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:11:48.573 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:48.586 INFO:tasks.rados.rados.0.smithi142.stdout:280: writing smithi14230322-42 from 704512 to 1376256 tid 1 2024-04-18T23:11:48.587 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:11:48.592 INFO:tasks.rados.rados.0.smithi142.stdout:280: writing smithi14230322-42 from 1376256 to 2129920 tid 2 2024-04-18T23:11:48.596 INFO:tasks.rados.rados.0.smithi142.stdout:280: writing smithi14230322-42 from 2129920 to 2539520 tid 3 2024-04-18T23:11:48.596 INFO:tasks.rados.rados.0.smithi142.stdout:281: snap_create 2024-04-18T23:11:48.596 INFO:tasks.rados.rados.0.smithi142.stdout:272: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-18T23:11:48.605 INFO:tasks.rados.rados.0.smithi142.stdout:274: finishing write tid 1 to smithi14230322-10 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:274: finishing write tid 2 to smithi14230322-10 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:274: finishing write tid 3 to smithi14230322-10 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:274: finishing write tid 4 to smithi14230322-10 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:274: finishing write tid 5 to smithi14230322-10 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:274: oid 10 updating version 0 to 102 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:274: oid 10 version 102 is already newer than 101 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:274: oid 10 updating version 102 to 104 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:274: oid 10 version 104 is already newer than 103 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 10 v 104 (ObjNum 107 snap 25 seq_num 107) dirty exists 2024-04-18T23:11:48.606 INFO:tasks.rados.rados.0.smithi142.stdout:274: left oid 10 (ObjNum 107 snap 25 seq_num 107) 2024-04-18T23:11:48.639 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:48.639+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:48.739 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:48.741+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:48.756 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:48.756 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:48.756 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:48.757 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:48.757 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:48.774 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:11:48.859 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:48.861+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:48.940 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:133 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'91, mlcod=28'85)", 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.663516295999997, 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20553276400000001, 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.941 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.942 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277424+0000", 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.21e-06 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277436+0000", 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1693e-05 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277464+0000", 2024-04-18T23:11:48.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8608000000000001e-05 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377595+0000", 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100130684 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377730+0000", 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013455499999999999 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377781+0000", 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1406999999999998e-05 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482870+0000", 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105089222 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482955+0000", 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4384999999999999e-05 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:155 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'109, mlcod=28'105)", 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.805978185999997, 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20593233399999999, 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.946 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134962+0000", 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8560000000000001e-06 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135074+0000", 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112357 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135099+0000", 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4705000000000001e-05 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135115+0000", 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5855999999999999e-05 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135494+0000", 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037909800000000002 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135551+0000", 2024-04-18T23:11:48.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7176999999999998e-05 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340852+0000", 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20530157900000001 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340892+0000", 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9706000000000002e-05 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.950 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:156 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'110, mlcod=28'105)", 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.805171559000001, 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.205157487, 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135766+0000", 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135768+0000", 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2130000000000001e-06 2024-04-18T23:11:48.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135779+0000", 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1749000000000001e-05 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135794+0000", 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4453e-05 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135807+0000", 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2599000000000001e-05 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136859+0000", 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010521199999999999 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136911+0000", 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2096999999999999e-05 2024-04-18T23:11:48.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340896+0000", 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20398571500000001 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340924+0000", 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7540999999999999e-05 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:157 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'111, mlcod=28'105)", 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 58.804456866999999, 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20611499799999999, 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:11:48.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136483+0000", 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.576e-06 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136499+0000", 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6529e-05 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136520+0000", 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0571999999999999e-05 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136532+0000", 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.223e-05 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237736+0000", 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10120357200000001 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:48.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237776+0000", 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0397e-05 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342561+0000", 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104784615 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342596+0000", 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5506999999999997e-05 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.960 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:201 3.es0 3:78aee268:::smithi14230322-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.379182448000002, 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41057925299999998, 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.961 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:11:48.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561757+0000", 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.47e-07 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561762+0000", 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.04e-06 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561777+0000", 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4962e-05 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561785+0000", 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4500000000000004e-06 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562118+0000", 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000332882 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562139+0000", 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1101999999999999e-05 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.769814+0000", 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.207675318 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:48.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770164+0000", 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034956199999999998 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.972335+0000", 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20217098999999999 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.966 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.4.21:1 3.es0 3:7a4badf6:::smithi14230322-17:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 48.179405615, 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31203379199999998, 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761535+0000", 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7999999999999999e-06 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761547+0000", 2024-04-18T23:11:48.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1279e-05 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761568+0000", 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1549e-05 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761588+0000", 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0426000000000001e-05 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770853+0000", 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092644660000000007 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.771105+0000", 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025160800000000001 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073566+0000", 2024-04-18T23:11:48.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30246166400000002 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:333 3.es0 3:799dcc49:::smithi14230322-39:head [create,setxattr Yjk]k``VRNVkfm]Rnjieeerecjk`Wm (15648) in=30b,setxattr QgpqijWTrohYWOsYikii[YdsSee`QrtrZ]nSPTUt (1225) in=40b,setxattr l]iRdbVhahjiOeVf`Yc\\U_OYjY]nr[js\\NggsVVY (1015) in=40b,setxattr b^W[ZgXWiRmfON^`rihabiO]lNkro`P (15648) in=31b,setxattr VReWeRiYgNSZU\\d\\YOrNS`]ogtdOjVijWVeY`fhd (1051) in=40b,setxattr SYsfepqPdsc`cUg\\WaTfgrcUYhUbTiaQY_`rlt (15648) in=38b,setxattr P[`jheVP[fjpnXqdjPhddi_oWUf\\dWWbSlodc`eS (1086) in=40b,setxattr X[^VkT`QVNea_jfmlVkdafY]hrorgVhl]fqtiVie (1226) in=40b,setxattr Xs\\WjfX^_Ss_RPr_tT^a]fnSSlYrcPTbmSaksi_c (19437) in=40b,setxattr QUn_^p]__nRhn\\srj]jla^iQSPOpgQcVcQZU^ool (1823) in=40b,setxattr f\\^WXdjheR\\aQSYk]Un_^hkrZTSVdhiZqgcin[Sa (1155) in=40b,setxattr bkb]mNeZddhQrcsORPSgqbWecOTtcPYZaRQOg`nY (1614) in=40b,setxattr _T_atc\\\\Rml^aOPkNsbsb^VatOWSs\\URSjVqfe\\Z (1226) in=40b,setxattr obYqlk]hgNOQgXa^ZhalmPcXl\\lkOdNNiYlOfdhq (1612) in=40b,setxattr VsbnoacTrXVpb]PV]kVPYbQP_qefUdfOgk`qUaNa (1999) in=40b,setxattr ZqTlaiiPajkNT^ce\\W\\oklalkiT`\\\\nhjcgpZrUa (3297) in=40b,setxattr QfUmVl\\][PRg\\ajWQS`]Oe^Zlig]b]Rs_o]NsY[i (244) in=40b,setxattr R_Ucr`a\\j]Ri_YSp[gamgRSRd`]UYnPRspctZ`R] (4068) in=40b,setxattr aj`m[hpi__Rrg_Z[m^rVOVrVgk^cS_]piWV]lkRZ (947) in=40b,setxattr ^nWmd_TeciTqUOnWXn_jXch]heNac[WdcqdPbiT^ (1647) in=40b] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.313582253, 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30962595399999998, 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.971 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627359+0000", 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3400000000000002e-06 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627676+0000", 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000317144 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627710+0000", 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3855e-05 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627753+0000", 2024-04-18T23:11:48.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3341999999999999e-05 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627926+0000", 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017248799999999999 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627946+0000", 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9868999999999999e-05 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729527+0000", 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101581587 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729747+0000", 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021955299999999999 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.733259+0000", 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035115820000000001 2024-04-18T23:11:48.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739147+0000", 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058886169999999996 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936726+0000", 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.197578422 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936982+0000", 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025615499999999999 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:344 3.es0 3:7a4badf6:::smithi14230322-17:head [delete] snapc 8=[7,4] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.299523696000001, 2024-04-18T23:11:48.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.295980827, 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 344 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641417+0000", 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2630000000000002e-06 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641427+0000", 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0674e-05 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641460+0000", 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2768999999999997e-05 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641482+0000", 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1959000000000001e-05 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641642+0000", 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016009100000000001 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641653+0000", 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.128e-05 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735170+0000", 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.093517063999999997 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735466+0000", 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029521400000000002 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.736750+0000", 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012846540000000001 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739533+0000", 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027831589999999999 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937242+0000", 2024-04-18T23:11:48.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19770827199999999 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937395+0000", 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000153428 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 56/49 ECSubWrite(tid=94, reqid=client.4568.0:421, at_version=56'16, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:48.985 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.297553276999999, 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.27351094799999998, 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.987 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643386+0000", 2024-04-18T23:11:48.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4300000000000002e-07 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643802+0000", 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000416136 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643934+0000", 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013215300000000001 2024-04-18T23:11:48.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643959+0000", 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4973e-05 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.812936+0000", 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.168977035 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.812952+0000", 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6419e-05 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916818+0000", 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10386601099999999 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916896+0000", 2024-04-18T23:11:48.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7378000000000004e-05 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 56/49 ECSubWrite(tid=95, reqid=client.4568.0:422, at_version=56'17, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.291938027, 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26796196300000003, 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.993 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:11:48.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649002+0000", 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8199999999999999e-06 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649359+0000", 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035705699999999998 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649390+0000", 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.146e-05 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749486+0000", 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100095925 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.814696+0000", 2024-04-18T23:11:48.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.065209634000000002 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.814713+0000", 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7082999999999999e-05 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916904+0000", 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10219086500000001 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916962+0000", 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8118999999999997e-05 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:467 3.es0 3:799dcc49:::smithi14230322-39:head [read 0~638976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.387296567, 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39041841700000002, 2024-04-18T23:11:48.998 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 467 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:48.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553642+0000", 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3000000000000001e-07 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553645+0000", 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.813e-06 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553650+0000", 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6840000000000001e-06 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553655+0000", 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654393+0000", 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100738487 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654401+0000", 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9340000000000006e-06 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.002 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.657013+0000", 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026115800000000001 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.657158+0000", 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014565499999999999 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.944060+0000", 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28690162600000002 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.003 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=42, last_complete=61'29, committed=1, applied=1))", 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.181483653000001, 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28503450299999999, 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759455+0000", 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7100000000000001e-07 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759459+0000", 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1760000000000003e-06 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759471+0000", 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1523e-05 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.859553+0000", 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10008174 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.044450+0000", 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18489760999999999 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.044489+0000", 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8683000000000002e-05 2024-04-18T23:11:49.007 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=44, last_complete=61'31, committed=1, applied=1))", 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.581735272, 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29027006799999999, 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359204+0000", 2024-04-18T23:11:49.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.282e-06 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359210+0000", 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6579999999999999e-06 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359227+0000", 2024-04-18T23:11:49.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7283999999999999e-05 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359257+0000", 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9643000000000001e-05 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649393+0000", 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29013631200000001 2024-04-18T23:11:49.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649473+0000", 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9888999999999994e-05 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=45, last_complete=61'33, committed=1, applied=1))", 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:11:49.012 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.580604032, 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28925603500000002, 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.013 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360335+0000", 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8999999999999995e-07 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360340+0000", 2024-04-18T23:11:49.014 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4160000000000003e-06 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360354+0000", 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3709999999999999e-05 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360362+0000", 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6120000000000001e-06 2024-04-18T23:11:49.015 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649559+0000", 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28919758400000001 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649590+0000", 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0822999999999999e-05 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.016 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:497 3.es0 3:799dcc49:::smithi14230322-39:head [rmxattr QgpqijWTrohYWOsYikii[YdsSee`QrtrZ]nSPTUt in=40b,rmxattr R_Ucr`a\\j]Ri_YSp[gamgRSRd`]UYnPRspctZ`R] in=40b,rmxattr VReWeRiYgNSZU\\d\\YOrNS`]ogtdOjVijWVeY`fhd in=40b,rmxattr X[^VkT`QVNea_jfmlVkdafY]hrorgVhl]fqtiVie in=40b,rmxattr _T_atc\\\\Rml^aOPkNsbsb^VatOWSs\\URSjVqfe\\Z in=40b,rmxattr bkb]mNeZddhQrcsORPSgqbWecOTtcPYZaRQOg`nY in=40b] snapc 15=[15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.410301093000001, 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21045935599999999, 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 497 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.017 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530641+0000", 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7869999999999999e-06 2024-04-18T23:11:49.018 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530672+0000", 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1642e-05 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530707+0000", 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4566000000000001e-05 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530726+0000", 2024-04-18T23:11:49.019 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9130000000000001e-05 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530864+0000", 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138307 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530878+0000", 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.379e-05 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.020 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.534931+0000", 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004053035 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.535122+0000", 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190612 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.536668+0000", 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015465800000000001 2024-04-18T23:11:49.021 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.640407+0000", 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103738334 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.740945+0000", 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100538086 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.022 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.741096+0000", 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015148700000000001 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:551 3.es0 3:78aee268:::smithi14230322-13:head [append 0~425984] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.563216+0000", 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.377721879999999, 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31486470599999999, 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.023 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 551 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.563216+0000", 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.024 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.563216+0000", 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.563219+0000", 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.92e-06 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564232+0000", 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010124229999999999 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.025 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564266+0000", 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4282000000000001e-05 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564286+0000", 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0134000000000001e-05 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564452+0000", 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000165876 2024-04-18T23:11:49.026 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564647+0000", 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019565699999999999 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.566327+0000", 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001679313 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.568134+0000", 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018070339999999999 2024-04-18T23:11:49.027 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.877907+0000", 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30977282 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878081+0000", 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017424699999999999 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.028 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:552 3.es0 3:78aee268:::smithi14230322-13:head [append 0~786432] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.570285+0000", 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.370652804000001, 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30865453999999998, 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:49.029 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 552 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.570285+0000", 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.570285+0000", 2024-04-18T23:11:49.030 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.570295+0000", 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3340000000000008e-06 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.571762+0000", 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001467007 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.031 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.571793+0000", 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1303000000000002e-05 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.571808+0000", 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4853e-05 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.670635+0000", 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098827154 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.032 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.670823+0000", 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018851999999999999 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.672550+0000", 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017261030000000001 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.777055+0000", 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10450546500000001 2024-04-18T23:11:49.033 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878818+0000", 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10176346999999999 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878940+0000", 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012133099999999999 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.034 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:553 3.es0 3:78aee268:::smithi14230322-13:head [append 0~376832] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.573318+0000", 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.367619574000001, 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40648862800000002, 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 553 2024-04-18T23:11:49.035 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.573318+0000", 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.573318+0000", 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.036 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.573320+0000", 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5239999999999999e-06 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574342+0000", 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010218429999999999 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574366+0000", 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3689999999999998e-05 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.037 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574378+0000", 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2386e-05 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.774530+0000", 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20015175199999999 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.774703+0000", 2024-04-18T23:11:49.038 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017317299999999999 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.776150+0000", 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014476020000000001 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878103+0000", 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101952266 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.039 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.979668+0000", 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101564918 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.979807+0000", 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139474 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.040 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:554 3.es0 3:78aee268:::smithi14230322-13:head [setxattr _header (57) in=7b] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.574400+0000", 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.366538137999999, 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50849109599999998, 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 554 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.041 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574400+0000", 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574400+0000", 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574401+0000", 2024-04-18T23:11:49.042 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.203e-06 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574407+0000", 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.207e-06 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574418+0000", 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0257e-05 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.043 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574425+0000", 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7319999999999998e-06 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.879281+0000", 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30485609800000002 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.879453+0000", 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017140699999999999 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.044 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.880371+0000", 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091843199999999995 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.881707+0000", 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001335943 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.082730+0000", 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201023233 2024-04-18T23:11:49.045 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.082891+0000", 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016058399999999999 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:555 3.es0 3:78aee268:::smithi14230322-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.574436+0000", 2024-04-18T23:11:49.046 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.366501569, 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81095218999999996, 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 555 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.047 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574436+0000", 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574436+0000", 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574437+0000", 2024-04-18T23:11:49.048 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7700000000000004e-07 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574441+0000", 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574451+0000", 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.012e-05 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.049 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.674559+0000", 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100107769 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.880898+0000", 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20633925 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.050 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.881048+0000", 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014995699999999999 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.083603+0000", 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202554858 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.083750+0000", 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014693 2024-04-18T23:11:49.051 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.385389+0000", 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30163906899999998 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.052 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:49.053 INFO:tasks.rados.rados.0.smithi142.stdout:280: finishing write tid 1 to smithi14230322-42 2024-04-18T23:11:49.054 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:11:49.055 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:49.051+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:49.065 INFO:tasks.rados.rados.0.smithi142.stdout:280: finishing write tid 2 to smithi14230322-42 2024-04-18T23:11:49.083 INFO:tasks.rados.rados.0.smithi142.stdout:278: finishing write tid 1 to smithi14230322-9 2024-04-18T23:11:49.083 INFO:tasks.rados.rados.0.smithi142.stdout:278: finishing write tid 2 to smithi14230322-9 2024-04-18T23:11:49.084 INFO:tasks.rados.rados.0.smithi142.stdout:278: finishing write tid 3 to smithi14230322-9 2024-04-18T23:11:49.084 INFO:tasks.rados.rados.0.smithi142.stdout:278: finishing write tid 4 to smithi14230322-9 2024-04-18T23:11:49.084 INFO:tasks.rados.rados.0.smithi142.stdout:278: finishing write tid 5 to smithi14230322-9 2024-04-18T23:11:49.154 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:49.155+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:49.166 INFO:tasks.rados.rados.0.smithi142.stdout:280: finishing write tid 3 to smithi14230322-42 2024-04-18T23:11:49.166 INFO:tasks.rados.rados.0.smithi142.stdout:280: finishing write tid 4 to smithi14230322-42 2024-04-18T23:11:49.170 INFO:tasks.rados.rados.0.smithi142.stdout:279: expect (ObjNum 29 snap 0 seq_num 29) 2024-04-18T23:11:49.228 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:49.228 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:49.228 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:11:49.228 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:49.242 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:11:49.255 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:49.255+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:49.289 INFO:tasks.rados.rados.0.smithi142.stdout:278: finishing write tid 6 to smithi14230322-9 2024-04-18T23:11:49.289 INFO:tasks.rados.rados.0.smithi142.stdout:278: oid 9 updating version 0 to 53 2024-04-18T23:11:49.289 INFO:tasks.rados.rados.0.smithi142.stdout:278: oid 9 updating version 53 to 54 2024-04-18T23:11:49.289 INFO:tasks.rados.rados.0.smithi142.stdout:278: oid 9 updating version 54 to 57 2024-04-18T23:11:49.289 INFO:tasks.rados.rados.0.smithi142.stdout:278: oid 9 version 57 is already newer than 55 2024-04-18T23:11:49.289 INFO:tasks.rados.rados.0.smithi142.stdout:278: oid 9 version 57 is already newer than 56 2024-04-18T23:11:49.289 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 9 v 57 (ObjNum 108 snap 25 seq_num 108) dirty exists 2024-04-18T23:11:49.290 INFO:tasks.rados.rados.0.smithi142.stdout:278: left oid 9 (ObjNum 108 snap 25 seq_num 108) 2024-04-18T23:11:49.356 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:49.357+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:49.366 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:49.366 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:49.366 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:49.369 INFO:tasks.rados.rados.0.smithi142.stdout:280: finishing write tid 5 to smithi14230322-42 2024-04-18T23:11:49.369 INFO:tasks.rados.rados.0.smithi142.stdout:280: oid 42 updating version 0 to 98 2024-04-18T23:11:49.369 INFO:tasks.rados.rados.0.smithi142.stdout:280: oid 42 updating version 98 to 99 2024-04-18T23:11:49.369 INFO:tasks.rados.rados.0.smithi142.stdout:280: oid 42 updating version 99 to 101 2024-04-18T23:11:49.369 INFO:tasks.rados.rados.0.smithi142.stdout:280: oid 42 version 101 is already newer than 100 2024-04-18T23:11:49.369 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 42 v 101 (ObjNum 109 snap 25 seq_num 109) dirty exists 2024-04-18T23:11:49.370 INFO:tasks.rados.rados.0.smithi142.stdout:280: left oid 42 (ObjNum 109 snap 25 seq_num 109) 2024-04-18T23:11:49.381 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:11:49.457 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:49.454+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:49.511 INFO:tasks.rados.rados.0.smithi142.stdout:272: done (5 left) 2024-04-18T23:11:49.511 INFO:tasks.rados.rados.0.smithi142.stdout:274: done (4 left) 2024-04-18T23:11:49.511 INFO:tasks.rados.rados.0.smithi142.stdout:278: done (3 left) 2024-04-18T23:11:49.511 INFO:tasks.rados.rados.0.smithi142.stdout:279: done (2 left) 2024-04-18T23:11:49.511 INFO:tasks.rados.rados.0.smithi142.stdout:280: done (1 left) 2024-04-18T23:11:49.511 INFO:tasks.rados.rados.0.smithi142.stdout:281: done (0 left) 2024-04-18T23:11:49.511 INFO:tasks.rados.rados.0.smithi142.stdout:282: delete oid 41 current snap is 26 2024-04-18T23:11:49.512 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.12 to 1 (10000) 2024-04-18T23:11:49.538 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-04-18T23:11:49.554 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=158, reqid=client.4568.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:11:49.554 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.925700695000003, 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203058724, 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.555 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627730+0000", 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.606e-06 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627841+0000", 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110692 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627889+0000", 2024-04-18T23:11:49.556 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8211000000000003e-05 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627919+0000", 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9961999999999999e-05 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629700+0000", 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001780963 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629730+0000", 2024-04-18T23:11:49.557 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0082000000000001e-05 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830736+0000", 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201005607 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830787+0000", 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1601000000000003e-05 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.558 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=159, reqid=client.4568.0:25, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.925426881999996, 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20284285199999999, 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.559 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628004+0000", 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0270000000000001e-06 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628137+0000", 2024-04-18T23:11:49.560 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013336000000000001 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628197+0000", 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0504000000000002e-05 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628221+0000", 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3037999999999999e-05 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.561 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728171+0000", 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099950707999999999 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728190+0000", 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8709e-05 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830795+0000", 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102605138 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.562 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830845+0000", 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0368000000000003e-05 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=160, reqid=client.4568.0:26, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.925172160999999, 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20264686400000001, 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.563 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628258+0000", 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628292+0000", 2024-04-18T23:11:49.564 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4054999999999998e-05 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628317+0000", 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5035999999999999e-05 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628338+0000", 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1058999999999999e-05 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728828+0000", 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100489521 2024-04-18T23:11:49.565 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728846+0000", 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8419000000000001e-05 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830850+0000", 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10200397 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830904+0000", 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3844000000000002e-05 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.566 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=161, reqid=client.4568.0:31, at_version=34'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.925038117, 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202594633, 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.567 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628392+0000", 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6000000000000004e-07 2024-04-18T23:11:49.568 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628541+0000", 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149386 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628572+0000", 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0975000000000001e-05 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628594+0000", 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1608999999999999e-05 2024-04-18T23:11:49.569 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829299+0000", 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20070527499999999 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829314+0000", 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4718999999999999e-05 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830915+0000", 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001601412 2024-04-18T23:11:49.570 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830986+0000", 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0697000000000004e-05 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=162, reqid=client.4568.0:32, at_version=34'6, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.635797+0000", 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.917632361999999, 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.195800061, 2024-04-18T23:11:49.571 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635797+0000", 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635797+0000", 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.572 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635799+0000", 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9039999999999999e-06 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635849+0000", 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0281000000000002e-05 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635897+0000", 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7416999999999997e-05 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.573 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.635923+0000", 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6486e-05 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830004+0000", 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19408114400000001 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830019+0000", 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4533e-05 2024-04-18T23:11:49.574 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.831518+0000", 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014992300000000001 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.831597+0000", 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9066000000000005e-05 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.575 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s5 42/31 ECSubWrite(tid=66, reqid=osd.14.0:65, at_version=0'0, trim_to=0'0, roll_forward_to=42'12))", 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:07.669347+0000", 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 41.884082632000002, 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20082834299999999, 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669347+0000", 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.576 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669347+0000", 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669348+0000", 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7400000000000002e-07 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669371+0000", 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3332e-05 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.577 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669458+0000", 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6646999999999999e-05 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.669484+0000", 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6706000000000001e-05 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.769505+0000", 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100021191 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.578 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.769528+0000", 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2132000000000001e-05 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.870118+0000", 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100590156 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.870175+0000", 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7404999999999997e-05 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.579 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 46/31 ECSubWrite(tid=20, reqid=client.4568.0:331, at_version=46'14, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.432076+0000", 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.121353427000003, 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20163852800000001, 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432076+0000", 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.580 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432076+0000", 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432077+0000", 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1399999999999997e-07 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432118+0000", 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1155999999999997e-05 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.581 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432134+0000", 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6198e-05 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432147+0000", 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3385e-05 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.632376+0000", 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200228505 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.582 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.632385+0000", 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2879999999999998e-06 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.633631+0000", 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001246268 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.633715+0000", 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3114000000000004e-05 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.583 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs4 47/31 ECSubWrite(tid=114, reqid=osd.4.0:114, at_version=47'33, trim_to=0'0, roll_forward_to=46'31))", 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.630014+0000", 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.923415775999999, 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20008867999999999, 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630014+0000", 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.584 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630014+0000", 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630014+0000", 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6600000000000002e-07 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630056+0000", 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1798999999999997e-05 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.585 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.630088+0000", 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2441999999999999e-05 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.730185+0000", 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100096952 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828149+0000", 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.097963921999999995 2024-04-18T23:11:49.586 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828173+0000", 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3941999999999999e-05 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830006+0000", 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001833243 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830102+0000", 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6014000000000006e-05 2024-04-18T23:11:49.587 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 56/31 ECSubWrite(tid=164, reqid=client.4568.0:431, at_version=56'73, trim_to=0'0, roll_forward_to=55'71))", 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.007917576000001, 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20358216400000001, 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.588 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7300000000000002e-07 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545582+0000", 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9449000000000005e-05 2024-04-18T23:11:49.589 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545607+0000", 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5741999999999998e-05 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545625+0000", 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7503000000000002e-05 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.645851+0000", 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100225733 2024-04-18T23:11:49.590 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.645870+0000", 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8972999999999999e-05 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749022+0000", 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103152813 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749094+0000", 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1577999999999998e-05 2024-04-18T23:11:49.591 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s6 70/68 ECSubWrite(tid=80, reqid=client.4568.0:549, at_version=0'0, trim_to=0'0, roll_forward_to=69'41))", 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:37.152355+0000", 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.401074585, 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200897031, 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152355+0000", 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.592 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152355+0000", 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152357+0000", 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.784e-06 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152384+0000", 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7650000000000001e-05 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.593 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152417+0000", 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2422000000000002e-05 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152440+0000", 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3260000000000001e-05 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152689+0000", 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024853199999999998 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.594 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152709+0000", 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0006000000000001e-05 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.353193+0000", 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200484738 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.595 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.353252+0000", 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8638999999999999e-05 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s6 70/68 ECSubWrite(tid=81, reqid=client.4568.0:550, at_version=70'42, trim_to=0'0, roll_forward_to=69'41))", 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:37.152485+0000", 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 12.400944819999999, 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201685279, 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.596 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152485+0000", 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152485+0000", 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152486+0000", 2024-04-18T23:11:49.597 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0999999999999997e-07 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152536+0000", 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0312000000000002e-05 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152560+0000", 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4046e-05 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.598 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152580+0000", 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9792999999999998e-05 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.352861+0000", 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20028146199999999 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.352877+0000", 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5420000000000001e-05 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.599 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.354095+0000", 2024-04-18T23:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012185469999999999 2024-04-18T23:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.354170+0000", 2024-04-18T23:11:49.600 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4789000000000005e-05 2024-04-18T23:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.601 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 71/70 ECSubWrite(tid=35, reqid=osd.5.0:35, at_version=71'26, trim_to=0'0, roll_forward_to=71'22))", 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.092364+0000", 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.461065407, 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20238825899999999, 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092364+0000", 2024-04-18T23:11:49.602 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092364+0000", 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092365+0000", 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.373e-06 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.603 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092404+0000", 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8241999999999997e-05 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092428+0000", 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4729999999999999e-05 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092443+0000", 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4883e-05 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.604 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.292777+0000", 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200333228 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.292791+0000", 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4059e-05 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.294674+0000", 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001883205 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.605 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.294752+0000", 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8539000000000001e-05 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 72/70 ECSubWrite(tid=40, reqid=osd.5.0:39, at_version=0'0, trim_to=0'0, roll_forward_to=72'30))", 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:39.022976+0000", 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.530453534999999, 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20188773400000001, 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.606 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.022976+0000", 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.022976+0000", 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.022978+0000", 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6169999999999999e-06 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.023000+0000", 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2660999999999999e-05 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.023038+0000", 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8104e-05 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.023054+0000", 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6062e-05 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.123257+0000", 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10020230199999999 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.123276+0000", 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8842999999999999e-05 2024-04-18T23:11:49.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.224763+0000", 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101487857 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.224864+0000", 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100288 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s5 72/71 ECSubWrite(tid=149, reqid=osd.14.0:149, at_version=72'26, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:39.024690+0000", 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.528739388, 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20278465600000001, 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024690+0000", 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024690+0000", 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024692+0000", 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.612 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024756+0000", 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4202000000000006e-05 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024796+0000", 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9762999999999999e-05 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024821+0000", 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4688000000000001e-05 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.223945+0000", 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19912484499999999 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.223959+0000", 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3985000000000001e-05 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.227409+0000", 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003449428 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.227475+0000", 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5932000000000005e-05 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s3 77/67 ECSubRead(tid=216, to_read={3:42c34e84:::smithi14230322-11:head=0,458752,0}, subchunks={3:42c34e84:::smithi14230322-11:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:43.842474+0000", 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.7109554879999997, 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20073938099999999, 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.615 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.842474+0000", 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.842474+0000", 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.842476+0000", 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8759999999999999e-06 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.842495+0000", 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9208999999999999e-05 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.842529+0000", 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4177999999999997e-05 2024-04-18T23:11:49.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.842556+0000", 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6724e-05 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.042963+0000", 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20040655499999999 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.043213+0000", 2024-04-18T23:11:49.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025083899999999999 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s3 77/67 ECSubRead(tid=217, to_read={3:42c34e84:::smithi14230322-11:head=0,458752,0}, subchunks={3:42c34e84:::smithi14230322-11:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:43.843070+0000", 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.7103592540000001, 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30067776099999999, 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.619 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843070+0000", 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843070+0000", 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843072+0000", 2024-04-18T23:11:49.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6169999999999999e-06 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843087+0000", 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843118+0000", 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0859999999999999e-05 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843143+0000", 2024-04-18T23:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4766e-05 2024-04-18T23:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.143457+0000", 2024-04-18T23:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30031421600000002 2024-04-18T23:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.143748+0000", 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029077999999999998 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 78/31 ECSubWrite(tid=259, reqid=osd.4.0:259, at_version=78'132, trim_to=0'0, roll_forward_to=77'130))", 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.613051+0000", 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9403783419999998, 2024-04-18T23:11:49.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20083701800000001, 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.613051+0000", 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.613051+0000", 2024-04-18T23:11:49.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.613052+0000", 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8000000000000001e-07 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.613095+0000", 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3080000000000001e-05 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.613119+0000", 2024-04-18T23:11:49.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4748000000000001e-05 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.713213+0000", 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100094092 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.713358+0000", 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014492300000000001 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.713388+0000", 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9462e-05 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.813823+0000", 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100435554 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.813888+0000", 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4778999999999998e-05 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 78/31 ECSubWrite(tid=264, reqid=osd.4.0:260, at_version=0'0, trim_to=0'0, roll_forward_to=78'134))", 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.818594+0000", 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.734835361, 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20360535399999999, 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818594+0000", 2024-04-18T23:11:49.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818594+0000", 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818596+0000", 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8870000000000001e-06 2024-04-18T23:11:49.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818629+0000", 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3408000000000003e-05 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818665+0000", 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5160000000000002e-05 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818689+0000", 2024-04-18T23:11:49.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4655e-05 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818836+0000", 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014703000000000001 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818853+0000", 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6636000000000001e-05 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022114+0000", 2024-04-18T23:11:49.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203261577 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022199+0000", 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5000999999999995e-05 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 78/31 ECSubWrite(tid=265, reqid=osd.4.0:265, at_version=78'136, trim_to=0'0, roll_forward_to=78'134))", 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.819697+0000", 2024-04-18T23:11:49.632 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.7337322229999996, 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202551706, 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819697+0000", 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819697+0000", 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819698+0000", 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.24e-06 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819756+0000", 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7926000000000001e-05 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819792+0000", 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5364000000000002e-05 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819815+0000", 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2861999999999999e-05 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.920043+0000", 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100228418 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.920060+0000", 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6769000000000002e-05 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022212+0000", 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10215207 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022249+0000", 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7057000000000001e-05 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s6 82/72 ECSubWrite(tid=35, reqid=osd.15.0:35, at_version=82'82, trim_to=0'0, roll_forward_to=81'80))", 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.680060+0000", 2024-04-18T23:11:49.636 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.87336958099999995, 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.194046933, 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.680060+0000", 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.680060+0000", 2024-04-18T23:11:49.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.680061+0000", 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9800000000000003e-07 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.680085+0000", 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4499999999999999e-05 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:49.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.680098+0000", 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3269e-05 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.780174+0000", 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10007559000000001 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.872273+0000", 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.092098797999999996 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:49.639 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.872285+0000", 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1962e-05 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.874033+0000", 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001747757 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.874107+0000", 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4258999999999995e-05 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.640 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:49.641 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:49.643 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:11:49.644 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:49.639+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:49.743 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:49.743+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:49.769 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:49.769 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:49.769 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:11:49.769 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:49.783 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:11:49.793 INFO:tasks.rados.rados.0.smithi142.stdout:282: done (0 left) 2024-04-18T23:11:49.793 INFO:tasks.rados.rados.0.smithi142.stdout:283: read oid 13 snap -1 2024-04-18T23:11:49.793 INFO:tasks.rados.rados.0.smithi142.stdout:283: expect (ObjNum 94 snap 21 seq_num 94) 2024-04-18T23:11:49.793 INFO:tasks.rados.rados.0.smithi142.stdout:284: setattr oid 26 current snap is 26 2024-04-18T23:11:49.795 INFO:tasks.rados.rados.0.smithi142.stdout:285: copy_from oid 32 from oid 7 current snap is 26 2024-04-18T23:11:49.795 INFO:tasks.rados.rados.0.smithi142.stdout:286: read oid 22 snap -1 2024-04-18T23:11:49.795 INFO:tasks.rados.rados.0.smithi142.stdout:286: expect deleted 2024-04-18T23:11:49.795 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 15 current snap is 26 2024-04-18T23:11:49.795 INFO:tasks.rados.rados.0.smithi142.stdout:287: seq_num 111 ranges {81920=704512,786432=655360,1441792=507904} 2024-04-18T23:11:49.804 INFO:tasks.rados.rados.0.smithi142.stdout:287: writing smithi14230322-15 from 81920 to 786432 tid 1 2024-04-18T23:11:49.811 INFO:tasks.rados.rados.0.smithi142.stdout:287: writing smithi14230322-15 from 786432 to 1441792 tid 2 2024-04-18T23:11:49.816 INFO:tasks.rados.rados.0.smithi142.stdout:287: writing smithi14230322-15 from 1441792 to 1949696 tid 3 2024-04-18T23:11:49.817 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 49 current snap is 26 2024-04-18T23:11:49.817 INFO:tasks.rados.rados.0.smithi142.stdout:285: finishing copy_from to smithi14230322-32 2024-04-18T23:11:49.817 INFO:tasks.rados.rados.0.smithi142.stdout:285: got expected ENOENT (src dne) 2024-04-18T23:11:49.817 INFO:tasks.rados.rados.0.smithi142.stdout:288: seq_num 112 ranges {65536=720896,786432=802816,1589248=360448} 2024-04-18T23:11:49.824 INFO:tasks.rados.rados.0.smithi142.stdout:288: writing smithi14230322-49 from 65536 to 786432 tid 1 2024-04-18T23:11:49.833 INFO:tasks.rados.rados.0.smithi142.stdout:288: writing smithi14230322-49 from 786432 to 1589248 tid 2 2024-04-18T23:11:49.837 INFO:tasks.rados.rados.0.smithi142.stdout:288: writing smithi14230322-49 from 1589248 to 1949696 tid 3 2024-04-18T23:11:49.837 INFO:tasks.rados.rados.0.smithi142.stdout:289: rollback oid 33 current snap is 26 2024-04-18T23:11:49.837 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 33 to 25 2024-04-18T23:11:49.837 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 30 current snap is 26 2024-04-18T23:11:49.837 INFO:tasks.rados.rados.0.smithi142.stdout:290: seq_num 113 ranges {491520=737280,1228800=557056,1785856=606208} 2024-04-18T23:11:49.844 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:49.838+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:49.850 INFO:tasks.rados.rados.0.smithi142.stdout:290: writing smithi14230322-30 from 491520 to 1228800 tid 1 2024-04-18T23:11:49.855 INFO:tasks.rados.rados.0.smithi142.stdout:290: writing smithi14230322-30 from 1228800 to 1785856 tid 2 2024-04-18T23:11:49.861 INFO:tasks.rados.rados.0.smithi142.stdout:290: writing smithi14230322-30 from 1785856 to 2392064 tid 3 2024-04-18T23:11:49.862 INFO:tasks.rados.rados.0.smithi142.stdout:291: read oid 37 snap -1 2024-04-18T23:11:49.862 INFO:tasks.rados.rados.0.smithi142.stdout:291: expect (ObjNum 85 snap 20 seq_num 85) 2024-04-18T23:11:49.862 INFO:tasks.rados.rados.0.smithi142.stdout:292: rollback oid 8 current snap is 26 2024-04-18T23:11:49.862 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 8 to 19 2024-04-18T23:11:49.862 INFO:tasks.rados.rados.0.smithi142.stdout:293: rmattr oid 46 current snap is 26 2024-04-18T23:11:49.862 INFO:tasks.rados.rados.0.smithi142.stdout:293: done (10 left) 2024-04-18T23:11:49.862 INFO:tasks.rados.rados.0.smithi142.stdout:294: copy_from oid 3 from oid 5 current snap is 26 2024-04-18T23:11:49.862 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 46 current snap is 26 2024-04-18T23:11:49.862 INFO:tasks.rados.rados.0.smithi142.stdout:295: seq_num 114 ranges {0=753664,753664=688128,1441792=475136} 2024-04-18T23:11:49.867 INFO:tasks.rados.rados.0.smithi142.stdout:295: writing smithi14230322-46 from 0 to 753664 tid 1 2024-04-18T23:11:49.872 INFO:tasks.rados.rados.0.smithi142.stdout:295: writing smithi14230322-46 from 753664 to 1441792 tid 2 2024-04-18T23:11:49.876 INFO:tasks.rados.rados.0.smithi142.stdout:295: writing smithi14230322-46 from 1441792 to 1916928 tid 3 2024-04-18T23:11:49.876 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 31 current snap is 26 2024-04-18T23:11:49.876 INFO:tasks.rados.rados.0.smithi142.stdout:296: seq_num 115 ranges {0=770048,770048=704512,1474560=458752} 2024-04-18T23:11:49.881 INFO:tasks.rados.rados.0.smithi142.stdout:296: writing smithi14230322-31 from 0 to 770048 tid 1 2024-04-18T23:11:49.886 INFO:tasks.rados.rados.0.smithi142.stdout:296: writing smithi14230322-31 from 770048 to 1474560 tid 2 2024-04-18T23:11:49.889 INFO:tasks.rados.rados.0.smithi142.stdout:296: writing smithi14230322-31 from 1474560 to 1933312 tid 3 2024-04-18T23:11:49.889 INFO:tasks.rados.rados.0.smithi142.stdout:297: read oid 12 snap 19 2024-04-18T23:11:49.889 INFO:tasks.rados.rados.0.smithi142.stdout:297: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-18T23:11:49.890 INFO:tasks.rados.rados.0.smithi142.stdout:298: rmattr oid 34 current snap is 26 2024-04-18T23:11:49.890 INFO:tasks.rados.rados.0.smithi142.stdout:298: done (14 left) 2024-04-18T23:11:49.890 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 38 current snap is 26 2024-04-18T23:11:49.890 INFO:tasks.rados.rados.0.smithi142.stdout:299: seq_num 116 ranges {0=786432,786432=458752,1245184=442368,1687552=262144} 2024-04-18T23:11:49.894 INFO:tasks.rados.rados.0.smithi142.stdout:299: writing smithi14230322-38 from 0 to 786432 tid 1 2024-04-18T23:11:49.897 INFO:tasks.rados.rados.0.smithi142.stdout:299: writing smithi14230322-38 from 786432 to 1245184 tid 2 2024-04-18T23:11:49.900 INFO:tasks.rados.rados.0.smithi142.stdout:299: writing smithi14230322-38 from 1245184 to 1687552 tid 3 2024-04-18T23:11:49.901 INFO:tasks.rados.rados.0.smithi142.stdout:299: writing smithi14230322-38 from 1687552 to 1949696 tid 4 2024-04-18T23:11:49.901 INFO:tasks.rados.rados.0.smithi142.stdout:300: read oid 34 snap -1 2024-04-18T23:11:49.902 INFO:tasks.rados.rados.0.smithi142.stdout:300: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-18T23:11:49.902 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:11:49.907 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:49.907 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:49.907 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:49.908 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:49.908 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:49.926 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:11:49.945 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:49.942+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:49.963 INFO:tasks.rados.rados.0.smithi142.stdout:286: done (15 left) 2024-04-18T23:11:49.963 INFO:tasks.rados.rados.0.smithi142.stdout:301: read oid 27 snap 25 2024-04-18T23:11:49.963 INFO:tasks.rados.rados.0.smithi142.stdout:301: expect (ObjNum 26 snap 0 seq_num 26) 2024-04-18T23:11:49.963 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:11:49.963 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 26 v 102 (ObjNum 65 snap 0 seq_num 1466258006) dirty exists 2024-04-18T23:11:49.963 INFO:tasks.rados.rados.0.smithi142.stdout:284: done (15 left) 2024-04-18T23:11:49.963 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 4 current snap is 26 2024-04-18T23:11:49.964 INFO:tasks.rados.rados.0.smithi142.stdout:302: seq_num 117 ranges {65536=425984,491520=606208,1097728=540672,1638400=409600} 2024-04-18T23:11:49.966 INFO:tasks.rados.rados.0.smithi142.stdout:302: writing smithi14230322-4 from 65536 to 491520 tid 1 2024-04-18T23:11:49.969 INFO:tasks.rados.rados.0.smithi142.stdout:302: writing smithi14230322-4 from 491520 to 1097728 tid 2 2024-04-18T23:11:49.972 INFO:tasks.rados.rados.0.smithi142.stdout:302: writing smithi14230322-4 from 1097728 to 1638400 tid 3 2024-04-18T23:11:49.975 INFO:tasks.rados.rados.0.smithi142.stdout:302: writing smithi14230322-4 from 1638400 to 2048000 tid 4 2024-04-18T23:11:49.975 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 16 2024-04-18T23:11:50.046 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:50.047+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:50.093 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:50.093 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:50.093 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 34/31 ECSubWrite(tid=1, reqid=client.4568.0:29, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.636644+0000", 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.456697544000001, 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10234691999999999, 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.094 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636644+0000", 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636644+0000", 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636646+0000", 2024-04-18T23:11:50.095 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.103e-06 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636689+0000", 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3464999999999998e-05 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636729+0000", 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0005000000000001e-05 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.096 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.636763+0000", 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3417999999999998e-05 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.737095+0000", 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100332765 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.737122+0000", 2024-04-18T23:11:50.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6355000000000001e-05 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.738921+0000", 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017988730000000001 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.738991+0000", 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9936000000000003e-05 2024-04-18T23:11:50.098 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=25, reqid=client.4568.0:185, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.443257897000002, 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103575165, 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650085+0000", 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8929999999999999e-06 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650164+0000", 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8852999999999999e-05 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650193+0000", 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9352e-05 2024-04-18T23:11:50.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750332+0000", 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100138779 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750511+0000", 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000178679 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.102 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750538+0000", 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328999999999999e-05 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753559+0000", 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030212080000000001 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753658+0000", 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9072000000000003e-05 2024-04-18T23:11:50.103 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=29, reqid=client.4568.0:202, at_version=36'17, trim_to=0'0, roll_forward_to=36'15))", 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:01.073962+0000", 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.019378948000004, 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10286168, 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.104 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073962+0000", 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073962+0000", 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073964+0000", 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6300000000000001e-06 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.074019+0000", 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5228e-05 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.074049+0000", 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0009e-05 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.174167+0000", 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100118289 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.174350+0000", 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018256199999999999 2024-04-18T23:11:50.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.174375+0000", 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5023000000000001e-05 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.176733+0000", 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023585030000000001 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.176824+0000", 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0435999999999999e-05 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 47/31 ECSubWrite(tid=48, reqid=osd.7.0:48, at_version=47'29, trim_to=0'0, roll_forward_to=46'25))", 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.846919+0000", 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.246422434000003, 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10250812300000001, 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.109 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.846919+0000", 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.846919+0000", 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.846920+0000", 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5340000000000001e-06 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.846994+0000", 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3783999999999997e-05 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.847023+0000", 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8535e-05 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.847058+0000", 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5101999999999997e-05 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947290+0000", 2024-04-18T23:11:50.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10023225099999999 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947320+0000", 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0395999999999998e-05 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.949355+0000", 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020347059999999998 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.949427+0000", 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1815000000000003e-05 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 61/31 ECSubWrite(tid=70, reqid=osd.7.0:70, at_version=61'48, trim_to=0'0, roll_forward_to=58'46))", 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.341400124, 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30382444400000003, 2024-04-18T23:11:50.114 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.68e-07 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751960+0000", 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8916e-05 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751973+0000", 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3030999999999999e-05 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751984+0000", 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0858e-05 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.853770+0000", 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101786394 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.853799+0000", 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8918999999999999e-05 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055725+0000", 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20192524000000001 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055766+0000", 2024-04-18T23:11:50.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0917999999999997e-05 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 61/31 ECSubWrite(tid=71, reqid=osd.7.0:71, at_version=61'50, trim_to=0'0, roll_forward_to=58'46))", 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 23.339856628, 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30231752299999998, 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.119 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2699999999999999e-07 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753510+0000", 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4779000000000001e-05 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753524+0000", 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4423999999999999e-05 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753534+0000", 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7089999999999997e-06 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.954802+0000", 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20126760199999999 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.954819+0000", 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7501999999999999e-05 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055781+0000", 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10096224299999999 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055802+0000", 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0636999999999999e-05 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 61/31 ECSubWrite(tid=74, reqid=osd.7.0:73, at_version=0'0, trim_to=0'0, roll_forward_to=61'52))", 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.667871+0000", 2024-04-18T23:11:50.124 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 22.425470384, 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10176123500000001, 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.667871+0000", 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.667871+0000", 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.667871+0000", 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.13e-07 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.667892+0000", 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0454000000000001e-05 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.667941+0000", 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8955000000000001e-05 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.667967+0000", 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5783e-05 2024-04-18T23:11:50.127 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.768202+0000", 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100235276 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.768219+0000", 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7606000000000001e-05 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.769390+0000", 2024-04-18T23:11:50.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00117073 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.769632+0000", 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024181800000000001 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 70/68 ECSubWrite(tid=140, reqid=client.4568.0:546, at_version=70'56, trim_to=0'0, roll_forward_to=69'54))", 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.849944+0000", 2024-04-18T23:11:50.129 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 13.243396736999999, 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10380012, 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.849944+0000", 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.849944+0000", 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.849947+0000", 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4739999999999999e-06 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.850099+0000", 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0001516 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.850140+0000", 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1832000000000002e-05 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.850167+0000", 2024-04-18T23:11:50.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6480000000000001e-05 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.950446+0000", 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100279335 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.950482+0000", 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6226000000000002e-05 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.953653+0000", 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031705079999999998 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.953745+0000", 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1664999999999997e-05 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.134 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.fs5 72/31 ECSubWrite(tid=59, reqid=osd.11.0:59, at_version=72'17, trim_to=0'0, roll_forward_to=61'15))", 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.533558+0000", 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.559783613, 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10226629600000001, 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.533558+0000", 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.533558+0000", 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.533558+0000", 2024-04-18T23:11:50.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2300000000000002e-07 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.533613+0000", 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4568999999999999e-05 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.533637+0000", 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4539999999999999e-05 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.533677+0000", 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9419000000000003e-05 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.633961+0000", 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100284807 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.633995+0000", 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3515e-05 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.635709+0000", 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001713682 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.635824+0000", 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00011534100000000001 2024-04-18T23:11:50.139 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 72/31 ECSubWrite(tid=63, reqid=osd.11.0:63, at_version=72'32, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.837897+0000", 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.255443977000001, 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203598052, 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837897+0000", 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837897+0000", 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837898+0000", 2024-04-18T23:11:50.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8700000000000005e-07 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837972+0000", 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4115000000000004e-05 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837995+0000", 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2589999999999999e-05 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838014+0000", 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8879e-05 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.938244+0000", 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100230344 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.938265+0000", 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1396e-05 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041423+0000", 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10315798700000001 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041495+0000", 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1953999999999996e-05 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.144 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 72/31 ECSubWrite(tid=64, reqid=client.4568.0:562, at_version=72'33, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.838056+0000", 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 11.25528523, 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203492644, 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838056+0000", 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838056+0000", 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838057+0000", 2024-04-18T23:11:50.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3399999999999997e-07 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838308+0000", 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025151500000000001 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838327+0000", 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8638999999999999e-05 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838339+0000", 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1950000000000001e-05 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.039312+0000", 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20097348100000001 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.039330+0000", 2024-04-18T23:11:50.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7672e-05 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041507+0000", 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021768360000000001 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041549+0000", 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1616999999999998e-05 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 73/31 ECSubWrite(tid=69, reqid=osd.11.0:69, at_version=73'37, trim_to=0'0, roll_forward_to=72'35))", 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:39.615831+0000", 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 10.477510447, 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102378499, 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.615831+0000", 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.615831+0000", 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.615833+0000", 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0600000000000002e-06 2024-04-18T23:11:50.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.615890+0000", 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7605000000000002e-05 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.615930+0000", 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9907000000000003e-05 2024-04-18T23:11:50.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.716029+0000", 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100099031 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.716211+0000", 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018145800000000001 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.716232+0000", 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1022e-05 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.718104+0000", 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001872184 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.718209+0000", 2024-04-18T23:11:50.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000105232 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 78/31 ECSubWrite(tid=107, reqid=osd.7.0:107, at_version=78'86, trim_to=0'0, roll_forward_to=77'84))", 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.726020+0000", 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3673209570000004, 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10314021499999999, 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.155 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726020+0000", 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726020+0000", 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726021+0000", 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7000000000000004e-07 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726077+0000", 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5779999999999998e-05 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726112+0000", 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4863999999999997e-05 2024-04-18T23:11:50.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726142+0000", 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9958e-05 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.826516+0000", 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10037412499999999 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.826550+0000", 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3797999999999997e-05 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.829081+0000", 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025316090000000002 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.829160+0000", 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9311e-05 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 78/31 ECSubWrite(tid=108, reqid=osd.7.0:108, at_version=78'88, trim_to=0'0, roll_forward_to=77'84))", 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.728301+0000", 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.3650405399999999, 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10172730200000001, 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.160 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.728301+0000", 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.728301+0000", 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.728303+0000", 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9230000000000001e-06 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.728373+0000", 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0862e-05 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.728404+0000", 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0701000000000003e-05 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.728427+0000", 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2694999999999999e-05 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.827668+0000", 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099241499999999996 2024-04-18T23:11:50.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.827701+0000", 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2585000000000001e-05 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.829965+0000", 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00226413 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.830028+0000", 2024-04-18T23:11:50.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2905999999999997e-05 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 78/72 ECSubWrite(tid=175, reqid=osd.14.0:175, at_version=78'82, trim_to=0'0, roll_forward_to=78'80))", 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:45.028035+0000", 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0653058089999998, 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104985259, 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.165 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028035+0000", 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028035+0000", 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028037+0000", 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.576e-06 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028115+0000", 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8271999999999999e-05 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028149+0000", 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3943999999999999e-05 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028173+0000", 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3875000000000001e-05 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.130996+0000", 2024-04-18T23:11:50.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102822806 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.131031+0000", 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5243999999999997e-05 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.132952+0000", 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019205369999999999 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.133021+0000", 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9004999999999994e-05 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 78/72 ECSubWrite(tid=176, reqid=osd.14.0:176, at_version=78'84, trim_to=0'0, roll_forward_to=78'80))", 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:45.030664+0000", 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 5.0626775669999997, 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10301552899999999, 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.170 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.030664+0000", 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.030664+0000", 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.030665+0000", 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7859999999999999e-06 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.030702+0000", 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6910999999999999e-05 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.030728+0000", 2024-04-18T23:11:50.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5952000000000002e-05 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.030744+0000", 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5792000000000001e-05 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.131993+0000", 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101249266 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.132021+0000", 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8121000000000001e-05 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.133589+0000", 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015672069999999999 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.133679+0000", 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0494000000000005e-05 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 82/72 ECSubWrite(tid=194, reqid=client.4568.0:674, at_version=82'98, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.661023+0000", 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4323183400000001, 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30022358900000001, 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.175 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661023+0000", 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661023+0000", 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661024+0000", 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661197+0000", 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017270400000000001 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661218+0000", 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1543e-05 2024-04-18T23:11:50.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661229+0000", 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0539e-05 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.858998+0000", 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.197769312 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.859008+0000", 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0365e-05 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961168+0000", 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10215980500000001 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961246+0000", 2024-04-18T23:11:50.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8355000000000005e-05 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 82/72 ECSubWrite(tid=195, reqid=client.4568.0:675, at_version=82'99, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.760788+0000", 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.332553511, 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20119051900000001, 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.180 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.760788+0000", 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.760788+0000", 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.760790+0000", 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1160000000000002e-06 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761152+0000", 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036268500000000001 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761191+0000", 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8374000000000002e-05 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761221+0000", 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0431999999999999e-05 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.859990+0000", 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098768442999999997 2024-04-18T23:11:50.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.860005+0000", 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5036e-05 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961955+0000", 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10195014400000001 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961978+0000", 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3289e-05 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 82/72 ECSubWrite(tid=196, reqid=client.4568.0:676, at_version=82'100, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.762849+0000", 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3304923820000001, 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199319412, 2024-04-18T23:11:50.185 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762849+0000", 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762849+0000", 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762851+0000", 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.81e-06 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.763087+0000", 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236512 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.763123+0000", 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6171000000000003e-05 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.763151+0000", 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7212e-05 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961317+0000", 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19816698099999999 2024-04-18T23:11:50.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961328+0000", 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0524e-05 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.962142+0000", 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081420799999999997 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.962168+0000", 2024-04-18T23:11:50.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5993999999999999e-05 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 82/31 ECSubWrite(tid=135, reqid=osd.7.0:135, at_version=82'112, trim_to=0'0, roll_forward_to=81'110))", 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:49.273828+0000", 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.81951306499999999, 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10223262599999999, 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.190 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.273828+0000", 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.273828+0000", 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.273829+0000", 2024-04-18T23:11:50.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2699999999999998e-07 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.273856+0000", 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6597999999999998e-05 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.273870+0000", 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4072e-05 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.273879+0000", 2024-04-18T23:11:50.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4730000000000006e-06 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.374049+0000", 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10016989699999999 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.374067+0000", 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7691999999999999e-05 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.375986+0000", 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001919652 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.376061+0000", 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4414999999999998e-05 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.194 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.195 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.195 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:50.196 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:50.194+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:50.196 INFO:tasks.rados.rados.0.smithi142.stdout:294: finishing copy_from to smithi14230322-3 2024-04-18T23:11:50.197 INFO:tasks.rados.rados.0.smithi142.stdout:294: got expected ENOENT (src dne) 2024-04-18T23:11:50.197 INFO:tasks.rados.rados.0.smithi142.stdout:283: expect (ObjNum 94 snap 21 seq_num 94) 2024-04-18T23:11:50.197 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:11:50.297 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:50.297+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:50.327 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.9s0 83/72 ECSubReadReply(tid=203, attrs_read=0))", 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.247128+0000", 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.081940211999999998, 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.081979162999999994, 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.247128+0000", 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.247128+0000", 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.247130+0000", 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6470000000000001e-06 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.247225+0000", 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4569999999999997e-05 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.247247+0000", 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2699000000000001e-05 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.247263+0000", 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5970000000000001e-05 2024-04-18T23:11:50.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.9s0 83/72 ECSubReadReply(tid=204, attrs_read=0))", 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.248328+0000", 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.080741104999999994, 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.080860291000000001, 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.248328+0000", 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.248328+0000", 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.248329+0000", 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.187e-06 2024-04-18T23:11:50.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.248428+0000", 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9147999999999993e-05 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.248442+0000", 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4314999999999999e-05 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.248452+0000", 2024-04-18T23:11:50.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8870000000000004e-06 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.9s0 83/72 ECSubReadReply(tid=204, attrs_read=0))", 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.269900+0000", 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.059168202000000003, 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.059357828000000001, 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:50.334 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.269900+0000", 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.269900+0000", 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.269903+0000", 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2199999999999999e-06 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.269987+0000", 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4771999999999994e-05 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.270005+0000", 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7144999999999999e-05 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.270017+0000", 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2493e-05 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.9s0 83/72 ECSubReadReply(tid=202, attrs_read=0))", 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.311898+0000", 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.017170758000000001, 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017428676000000001, 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.337 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.311898+0000", 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.311898+0000", 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.311900+0000", 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1490000000000001e-06 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.312215+0000", 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031491199999999998 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.312248+0000", 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3461999999999999e-05 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.312272+0000", 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3958e-05 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpReadReply(3.9s0 83/72 ECSubReadReply(tid=203, attrs_read=0))", 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.312323+0000", 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.016745579999999999, 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.017071020999999999, 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.340 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.312323+0000", 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.312323+0000", 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.312324+0000", 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.0699999999999996e-07 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.312408+0000", 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4351999999999995e-05 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.312425+0000", 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6714999999999998e-05 2024-04-18T23:11:50.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.312439+0000", 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3593e-05 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:738 3.9s0 3:906aad0a:::smithi14230322-27:1b [read 0~442368,checksum in=4b,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:49.965692+0000", 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.36337693300000001, 2024-04-18T23:11:50.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.36378944899999999, 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 738 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.965692+0000", 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.965692+0000", 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.965693+0000", 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.1999999999999998e-07 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.965699+0000", 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1449999999999996e-06 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.965712+0000", 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2894999999999999e-05 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065816+0000", 2024-04-18T23:11:50.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100104335 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065977+0000", 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016137000000000001 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.066199+0000", 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00022164400000000001 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:739 3.9s0 3:906aad0a:::smithi14230322-27:1b [read 0~442368,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.065876+0000", 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.26319292799999999, 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26371618400000002, 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 739 2024-04-18T23:11:50.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065876+0000", 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065876+0000", 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065878+0000", 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3429999999999998e-06 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065893+0000", 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5166e-05 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065921+0000", 2024-04-18T23:11:50.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7868000000000002e-05 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065939+0000", 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8269000000000001e-05 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.066836+0000", 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00089678100000000003 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.066978+0000", 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000142182 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:740 3.9s0 3:906aad0a:::smithi14230322-27:1b [read 0~442368,checksum in=4b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.065970+0000", 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.263098687, 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26375901099999999, 2024-04-18T23:11:50.352 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 740 2024-04-18T23:11:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065970+0000", 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065970+0000", 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065972+0000", 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1129999999999999e-06 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.065981+0000", 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0429999999999996e-06 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.066002+0000", 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0486e-05 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.066017+0000", 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5034999999999999e-05 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.168754+0000", 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102737252 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.168828+0000", 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.3756000000000003e-05 2024-04-18T23:11:50.356 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-18T23:11:50.357 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 8 2024-04-18T23:11:50.357 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:50.358 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:11:50.371 INFO:tasks.rados.rados.0.smithi142.stdout:285: finishing copy_from racing read to smithi14230322-32 2024-04-18T23:11:50.371 INFO:tasks.rados.rados.0.smithi142.stdout:294: finishing copy_from racing read to smithi14230322-3 2024-04-18T23:11:50.371 INFO:tasks.rados.rados.0.smithi142.stdout:292: finishing rollback tid 0 to smithi14230322-8 2024-04-18T23:11:50.371 INFO:tasks.rados.rados.0.smithi142.stdout:292: finishing rollback tid 1 to smithi14230322-8 2024-04-18T23:11:50.371 INFO:tasks.rados.rados.0.smithi142.stdout:283: done (15 left) 2024-04-18T23:11:50.371 INFO:tasks.rados.rados.0.smithi142.stdout:285: done (14 left) 2024-04-18T23:11:50.372 INFO:tasks.rados.rados.0.smithi142.stdout:294: done (13 left) 2024-04-18T23:11:50.372 INFO:tasks.rados.rados.0.smithi142.stdout:303: snap_remove snap 21 2024-04-18T23:11:50.398 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:50.394+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:50.439 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:50.499 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:50.494+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:50.517 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:50.517 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:11:50.517 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:11:50.517 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:11:50.517 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:50.531 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:11:50.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T23:11:50.594 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:50.594 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:50.599 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:50.599+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:50.615 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:50.615 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:50.634 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:50.635 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:50.644 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.13 to 1 (10000) 2024-04-18T23:11:50.646 INFO:tasks.rados.rados.0.smithi142.stdout:303: done (13 left) 2024-04-18T23:11:50.646 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 44 current snap is 26 2024-04-18T23:11:50.646 INFO:tasks.rados.rados.0.smithi142.stdout:304: seq_num 118 ranges {0=442368,442368=753664,1196032=655360,1851392=147456} 2024-04-18T23:11:50.649 INFO:tasks.rados.rados.0.smithi142.stdout:304: writing smithi14230322-44 from 0 to 442368 tid 1 2024-04-18T23:11:50.655 INFO:tasks.rados.rados.0.smithi142.stdout:304: writing smithi14230322-44 from 442368 to 1196032 tid 2 2024-04-18T23:11:50.656 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:11:50.656 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:11:50.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:11:50.656 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:11:50.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.657 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:273 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:50.657 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:11:50.657 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.074707472, 2024-04-18T23:11:50.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.781474428, 2024-04-18T23:11:50.657 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.657 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.657 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:50.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 273 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581592+0000", 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999999e-06 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581611+0000", 2024-04-18T23:11:50.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8736000000000002e-05 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581652+0000", 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1421999999999997e-05 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681775+0000", 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100122909 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681951+0000", 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000175358 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681971+0000", 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0795e-05 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.858775+0000", 2024-04-18T23:11:50.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17680337099999999 2024-04-18T23:11:50.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.858953+0000", 2024-04-18T23:11:50.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017833000000000001 2024-04-18T23:11:50.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.363063+0000", 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50410986700000004 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.663 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:274 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.974461363000003, 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.68252750799999995, 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 274 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.664 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681837+0000", 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.756e-06 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681852+0000", 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4566000000000001e-05 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681881+0000", 2024-04-18T23:11:50.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8651999999999999e-05 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.781982+0000", 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100101015 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782144+0000", 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162347 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782164+0000", 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9888999999999999e-05 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.959687+0000", 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17752280100000001 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.959899+0000", 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021188899999999999 2024-04-18T23:11:50.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364362+0000", 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40446359300000001 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:275 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.874276948999999, 2024-04-18T23:11:50.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.78490300599999996, 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 275 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:11:50.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782021+0000", 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.497e-06 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782031+0000", 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8730000000000002e-06 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782049+0000", 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8016000000000002e-05 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782060+0000", 2024-04-18T23:11:50.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1066e-05 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782264+0000", 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020450899999999999 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782285+0000", 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1066000000000001e-05 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364492+0000", 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58220711199999997 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364799+0000", 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030637600000000002 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.566922+0000", 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20212349099999999 2024-04-18T23:11:50.674 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:341 3.9s0 3:906aad0a:::smithi14230322-27:head [append 0~0] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.724969+0000", 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.931327324999998, 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.60935175900000005, 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.675 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 341 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724969+0000", 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724969+0000", 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724971+0000", 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4899999999999999e-06 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724985+0000", 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.418e-05 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725014+0000", 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9016000000000001e-05 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725034+0000", 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9825000000000001e-05 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725162+0000", 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012797000000000001 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725175+0000", 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2255999999999999e-05 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828411+0000", 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103236438 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828613+0000", 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020252399999999999 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830448+0000", 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00183472 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.932371+0000", 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101922922 2024-04-18T23:11:50.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.334124+0000", 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40175268400000003 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.334321+0000", 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000196734 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:342 3.9s0 3:906aad0a:::smithi14230322-27:head [rollback 4] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.725072+0000", 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.931224018000002, 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71010575600000003, 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 342 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.682 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725072+0000", 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725072+0000", 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725074+0000", 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725081+0000", 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9959999999999996e-06 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725098+0000", 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7785999999999999e-05 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725112+0000", 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.403e-05 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725303+0000", 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019035999999999999 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725342+0000", 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8769000000000001e-05 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.928317+0000", 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20297505199999999 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.928568+0000", 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025112799999999998 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.929483+0000", 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000915371 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.932373+0000", 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002890218 2024-04-18T23:11:50.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.434969+0000", 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50259540999999996 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435178+0000", 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209156 2024-04-18T23:11:50.688 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:343 3.9s0 3:906aad0a:::smithi14230322-27:head [append 0~0] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.725142+0000", 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.931153934000001, 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71044070299999995, 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:50.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 343 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725142+0000", 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725142+0000", 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725144+0000", 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.347e-06 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725169+0000", 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5683000000000001e-05 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725187+0000", 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8235000000000001e-05 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725203+0000", 2024-04-18T23:11:50.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5933000000000001e-05 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725399+0000", 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019563999999999999 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725408+0000", 2024-04-18T23:11:50.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0999999999999993e-06 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.930345+0000", 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.204937228 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.930628+0000", 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000282274 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.931715+0000", 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010869009999999999 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.233078+0000", 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30136315400000002 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435450+0000", 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20237229500000001 2024-04-18T23:11:50.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435583+0000", 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013291299999999999 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 47/31 ECSubWriteReply(tid=80, last_complete=47'32, committed=1, applied=1))", 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.947474+0000", 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.708822144999999, 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.48772908199999998, 2024-04-18T23:11:50.696 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947474+0000", 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947474+0000", 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947475+0000", 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1829999999999999e-06 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947482+0000", 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.7870000000000004e-06 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.947500+0000", 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7669e-05 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.047588+0000", 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.1000887 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.699 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.434720+0000", 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.38713160200000002 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435203+0000", 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00048314099999999997 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:421 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~557056] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.533981+0000", 2024-04-18T23:11:50.700 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.122314873000001, 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81687969000000005, 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 421 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.533981+0000", 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.533981+0000", 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.533984+0000", 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2024-04-18T23:11:50.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.534844+0000", 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085963299999999995 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.534879+0000", 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5305999999999997e-05 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.534898+0000", 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2024-04-18T23:11:50.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.535053+0000", 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015493200000000001 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.535070+0000", 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6682999999999999e-05 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.704 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.640162+0000", 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105092212 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.640474+0000", 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031151200000000001 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643548+0000", 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003074181 2024-04-18T23:11:50.705 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.652446+0000", 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0088978619999999994 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350532+0000", 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.69808661400000005 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.706 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350861+0000", 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000328585 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:422 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~671744] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.537766+0000", 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.118529624000001, 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81361762999999998, 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.707 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 422 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.537766+0000", 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.708 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.537766+0000", 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.537769+0000", 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1059999999999998e-06 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538649+0000", 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088010300000000003 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.709 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538679+0000", 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0263000000000001e-05 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538694+0000", 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4936000000000001e-05 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538836+0000", 2024-04-18T23:11:50.710 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141643 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538849+0000", 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3013000000000001e-05 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.646583+0000", 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107734542 2024-04-18T23:11:50.711 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.646882+0000", 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029901899999999998 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.648879+0000", 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019973460000000001 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.712 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.653818+0000", 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049380880000000002 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.351099+0000", 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.69728165900000005 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.351384+0000", 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028491200000000001 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.713 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:423 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~98304] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.538732+0000", 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.117564130000002, 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.91325199599999995, 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.714 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 423 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538732+0000", 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538732+0000", 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538733+0000", 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538808+0000", 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4623999999999996e-05 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538822+0000", 2024-04-18T23:11:50.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.401e-05 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538832+0000", 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538898+0000", 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6571999999999999e-05 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:50.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538907+0000", 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.003e-06 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.650070+0000", 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111163027 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.650383+0000", 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000312844 2024-04-18T23:11:50.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.651521+0000", 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001137817 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.654205+0000", 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026839479999999998 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:50.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.451402+0000", 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.79719673599999996 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.451984+0000", 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00058231199999999998 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:424 3.0s0 3:063b0705:::smithi14230322-16:head [setxattr _header (57) in=7b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.538853+0000", 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.117442839999999, 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.014298884, 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.721 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 424 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538853+0000", 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538853+0000", 2024-04-18T23:11:50.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538854+0000", 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7300000000000004e-07 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538859+0000", 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6299999999999997e-06 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538869+0000", 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0302999999999999e-05 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538881+0000", 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2196e-05 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538998+0000", 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116821 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.539024+0000", 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5392e-05 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.652483+0000", 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.113459882 2024-04-18T23:11:50.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.652660+0000", 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000176522 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.653461+0000", 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000801299 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.655473+0000", 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020117630000000002 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.552510+0000", 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.89703736599999995 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.727 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.553152+0000", 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00064173699999999997 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:425 3.0s0 3:063b0705:::smithi14230322-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.538897+0000", 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.117399213999999, 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.219395432, 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.728 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 425 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538897+0000", 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538897+0000", 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538898+0000", 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8599999999999997e-07 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538901+0000", 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6169999999999998e-06 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538912+0000", 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0412e-05 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538920+0000", 2024-04-18T23:11:50.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5569999999999996e-06 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.638222+0000", 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099301983999999996 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.638243+0000", 2024-04-18T23:11:50.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0825000000000001e-05 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.654027+0000", 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015784231999999999 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.654189+0000", 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000161686 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.654544+0000", 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.000355017 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.654740+0000", 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019602899999999999 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.758292+0000", 2024-04-18T23:11:50.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10355228700000001 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=94, last_complete=56'16, committed=1, applied=1))", 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.752292+0000", 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.904004109999999, 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497650382, 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.735 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752292+0000", 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752292+0000", 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752294+0000", 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.767e-06 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752304+0000", 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9520000000000006e-06 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752329+0000", 2024-04-18T23:11:50.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852428+0000", 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100098916 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.249913+0000", 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39748424700000001 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.249942+0000", 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9757e-05 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=95, last_complete=56'17, committed=1, applied=1))", 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.852480+0000", 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.803815617000001, 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49763560899999998, 2024-04-18T23:11:50.739 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852480+0000", 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852480+0000", 2024-04-18T23:11:50.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852482+0000", 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0470000000000001e-06 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852489+0000", 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7100000000000007e-06 2024-04-18T23:11:50.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852510+0000", 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1109000000000001e-05 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852521+0000", 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.11e-05 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350081+0000", 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49755995800000002 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350116+0000", 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4684999999999997e-05 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=96, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:11:50.743 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.803747065, 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49767800099999998, 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:11:50.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000001e-07 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852553+0000", 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8999999999999999e-06 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852566+0000", 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3063e-05 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852574+0000", 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5440000000000001e-06 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350198+0000", 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497624179 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350227+0000", 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8901999999999999e-05 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=97, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.853074+0000", 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.803222363, 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49724826700000002, 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853074+0000", 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853074+0000", 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853075+0000", 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999999999999999e-06 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853082+0000", 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1999999999999997e-06 2024-04-18T23:11:50.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853100+0000", 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357000000000001e-05 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853111+0000", 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0389e-05 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350293+0000", 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49718215799999999 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350322+0000", 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8963e-05 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=96, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.917214+0000", 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.739082017000001, 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53479636500000005, 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917214+0000", 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917214+0000", 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917215+0000", 2024-04-18T23:11:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:11:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917220+0000", 2024-04-18T23:11:50.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7299999999999996e-06 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917235+0000", 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5273000000000001e-05 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917244+0000", 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0359999999999995e-06 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.451137+0000", 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53389361899999999 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.452010+0000", 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087298699999999998 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=97, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.917619+0000", 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.738676878, 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.636734154, 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:11:50.757 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917619+0000", 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917619+0000", 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917620+0000", 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9399999999999993e-07 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917627+0000", 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8360000000000003e-06 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917645+0000", 2024-04-18T23:11:50.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7566000000000001e-05 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.017700+0000", 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100055057 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.552241+0000", 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53454112899999995 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.554353+0000", 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021125720000000001 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:676 3.9s0 3:9e926f5e:::smithi14230322-42:head [append 0~409600] snapc 1a=[19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.599192+0000", 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 2.057103836, 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.56959064999999998, 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.761 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 676 2024-04-18T23:11:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599192+0000", 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599192+0000", 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599194+0000", 2024-04-18T23:11:50.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2170000000000001e-06 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599700+0000", 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050507899999999997 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599731+0000", 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1477999999999999e-05 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699847+0000", 2024-04-18T23:11:50.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100115674 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761918+0000", 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.062071227 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762022+0000", 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010433999999999999 2024-04-18T23:11:50.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762761+0000", 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073854300000000003 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.964670+0000", 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201908694 2024-04-18T23:11:50.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.168699+0000", 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20402904999999999 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.168783+0000", 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4347999999999999e-05 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:678 3.9s0 3:9e926f5e:::smithi14230322-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.699915+0000", 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.9563810799999999, 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.67171341299999998, 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 678 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:11:50.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699915+0000", 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699915+0000", 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699915+0000", 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9000000000000002e-07 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699918+0000", 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0990000000000001e-06 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699923+0000", 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3510000000000001e-06 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:11:50.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699927+0000", 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2339999999999996e-06 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.864219+0000", 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.16429192200000001 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.864302+0000", 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2857999999999998e-05 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:11:50.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.169513+0000", 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30521089099999998 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.169614+0000", 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101147 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.371628+0000", 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202014521 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:11:50.773 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:11:50.773 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:11:50.776 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-04-18T23:11:50.776 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:50.777 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:11:50.778 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:50.773+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:50.778 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:50.779 INFO:tasks.rados.rados.0.smithi142.stdout:304: writing smithi14230322-44 from 1196032 to 1851392 tid 3 2024-04-18T23:11:50.779 INFO:tasks.rados.rados.0.smithi142.stdout:304: writing smithi14230322-44 from 1851392 to 1998848 tid 4 2024-04-18T23:11:50.779 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 32 current snap is 26 2024-04-18T23:11:50.779 INFO:tasks.rados.rados.0.smithi142.stdout:292: finishing rollback tid 2 to smithi14230322-8 2024-04-18T23:11:50.779 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 8 v 78 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:296: finishing write tid 1 to smithi14230322-31 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:305: seq_num 119 ranges {1490944=458752,1949696=507904,2457600=507904,2965504=458752,3424256=81920} 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:305: writing smithi14230322-32 from 1490944 to 1949696 tid 1 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:305: writing smithi14230322-32 from 1949696 to 2457600 tid 2 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:305: writing smithi14230322-32 from 2457600 to 2965504 tid 3 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:305: writing smithi14230322-32 from 2965504 to 3424256 tid 4 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:305: writing smithi14230322-32 from 3424256 to 3506176 tid 5 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:292: done (14 left) 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:306: rollback oid 3 current snap is 26 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 3 to 17 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:300: expect (ObjNum 33 snap 0 seq_num 33) 2024-04-18T23:11:50.780 INFO:tasks.rados.rados.0.smithi142.stdout:296: finishing write tid 2 to smithi14230322-31 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:296: finishing write tid 3 to smithi14230322-31 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:296: finishing write tid 4 to smithi14230322-31 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:300: done (14 left) 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:307: rollback oid 13 current snap is 26 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 13 to 25 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:288: finishing write tid 1 to smithi14230322-49 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:301: expect (ObjNum 26 snap 0 seq_num 26) 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:289: finishing rollback tid 0 to smithi14230322-33 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:301: done (14 left) 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:308: rmattr oid 50 current snap is 26 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:308: done (14 left) 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:309: rollback oid 5 current snap is 26 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 5 to 11 2024-04-18T23:11:50.781 INFO:tasks.rados.rados.0.smithi142.stdout:287: finishing write tid 1 to smithi14230322-15 2024-04-18T23:11:50.782 INFO:tasks.rados.rados.0.smithi142.stdout:287: finishing write tid 2 to smithi14230322-15 2024-04-18T23:11:50.782 INFO:tasks.rados.rados.0.smithi142.stdout:287: finishing write tid 3 to smithi14230322-15 2024-04-18T23:11:50.782 INFO:tasks.rados.rados.0.smithi142.stdout:287: finishing write tid 4 to smithi14230322-15 2024-04-18T23:11:50.782 INFO:tasks.rados.rados.0.smithi142.stdout:291: expect (ObjNum 85 snap 20 seq_num 85) 2024-04-18T23:11:50.809 INFO:tasks.rados.rados.0.smithi142.stdout:297: expect (ObjNum 7 snap 0 seq_num 7) 2024-04-18T23:11:50.813 INFO:tasks.rados.rados.0.smithi142.stdout:290: finishing write tid 1 to smithi14230322-30 2024-04-18T23:11:50.813 INFO:tasks.rados.rados.0.smithi142.stdout:291: done (14 left) 2024-04-18T23:11:50.813 INFO:tasks.rados.rados.0.smithi142.stdout:297: done (13 left) 2024-04-18T23:11:50.813 INFO:tasks.rados.rados.0.smithi142.stdout:310: rmattr oid 18 current snap is 26 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:310: done (13 left) 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:311: copy_from oid 27 from oid 7 current snap is 26 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:290: finishing write tid 2 to smithi14230322-30 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:289: finishing rollback tid 1 to smithi14230322-33 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:289: finishing rollback tid 2 to smithi14230322-33 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 33 v 88 (ObjNum 57 snap 4 seq_num 57) dirty exists 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:290: finishing write tid 3 to smithi14230322-30 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:290: finishing write tid 4 to smithi14230322-30 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:295: finishing write tid 1 to smithi14230322-46 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:295: finishing write tid 2 to smithi14230322-46 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:295: finishing write tid 3 to smithi14230322-46 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:296: finishing write tid 5 to smithi14230322-31 2024-04-18T23:11:50.814 INFO:tasks.rados.rados.0.smithi142.stdout:296: oid 31 updating version 0 to 39 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:296: oid 31 version 39 is already newer than 38 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:296: oid 31 version 39 is already newer than 37 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:296: oid 31 updating version 39 to 40 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 31 v 40 (ObjNum 115 snap 26 seq_num 115) dirty exists 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:296: left oid 31 (ObjNum 115 snap 26 seq_num 115) 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:299: finishing write tid 1 to smithi14230322-38 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:288: finishing write tid 2 to smithi14230322-49 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:288: finishing write tid 3 to smithi14230322-49 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:288: finishing write tid 4 to smithi14230322-49 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:287: finishing write tid 5 to smithi14230322-15 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:287: oid 15 updating version 0 to 50 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:287: oid 15 updating version 50 to 53 2024-04-18T23:11:50.815 INFO:tasks.rados.rados.0.smithi142.stdout:287: oid 15 version 53 is already newer than 52 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:287: oid 15 version 53 is already newer than 51 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 15 v 53 (ObjNum 111 snap 26 seq_num 111) dirty exists 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:287: left oid 15 (ObjNum 111 snap 26 seq_num 111) 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:295: finishing write tid 4 to smithi14230322-46 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:299: finishing write tid 2 to smithi14230322-38 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:299: finishing write tid 3 to smithi14230322-38 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:299: finishing write tid 4 to smithi14230322-38 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:299: finishing write tid 5 to smithi14230322-38 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:290: finishing write tid 5 to smithi14230322-30 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:290: oid 30 updating version 0 to 118 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:290: oid 30 updating version 118 to 121 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:290: oid 30 version 121 is already newer than 120 2024-04-18T23:11:50.816 INFO:tasks.rados.rados.0.smithi142.stdout:290: oid 30 version 121 is already newer than 119 2024-04-18T23:11:50.817 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 30 v 121 (ObjNum 113 snap 26 seq_num 113) dirty exists 2024-04-18T23:11:50.817 INFO:tasks.rados.rados.0.smithi142.stdout:290: left oid 30 (ObjNum 113 snap 26 seq_num 113) 2024-04-18T23:11:50.817 INFO:tasks.rados.rados.0.smithi142.stdout:287: done (13 left) 2024-04-18T23:11:50.817 INFO:tasks.rados.rados.0.smithi142.stdout:289: done (12 left) 2024-04-18T23:11:50.817 INFO:tasks.rados.rados.0.smithi142.stdout:290: done (11 left) 2024-04-18T23:11:50.817 INFO:tasks.rados.rados.0.smithi142.stdout:296: done (10 left) 2024-04-18T23:11:50.817 INFO:tasks.rados.rados.0.smithi142.stdout:312: rollback oid 1 current snap is 26 2024-04-18T23:11:50.817 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 1 to 23 2024-04-18T23:11:50.817 INFO:tasks.rados.rados.0.smithi142.stdout:313: snap_remove snap 13 2024-04-18T23:11:50.874 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:50.870+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:50.910 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:50.910 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:50.924 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:11:50.976 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:50.970+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:51.075 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:51.075 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:51.076 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:51.077+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:51.099 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:51.099 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:51.110 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:51.111 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:51.114 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:11:51.139 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:51.139 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:51.159 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:51.159 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:51.177 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:51.175+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:51.243 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:51.244 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:51.257 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:11:51.278 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:51.277+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:51.299 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:51.300 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:51.323 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:51.324 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:51.344 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:51.345 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:51.368 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:51.368 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:51.379 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:51.381+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:51.390 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:51.390 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:51.413 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:51.413 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:51.422 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:51.422 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:51.436 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:11:51.437 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:51.437 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:51.480 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:51.474+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:51.494 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:51.495 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:51.558 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:51.558 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:51.572 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:11:51.580 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:51.578+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:313: done (11 left) 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:314: snap_create 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:302: finishing write tid 1 to smithi14230322-4 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:302: finishing write tid 2 to smithi14230322-4 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:302: finishing write tid 3 to smithi14230322-4 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:307: finishing rollback tid 0 to smithi14230322-13 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:307: finishing rollback tid 1 to smithi14230322-13 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:295: finishing write tid 5 to smithi14230322-46 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:295: oid 46 updating version 0 to 89 2024-04-18T23:11:51.658 INFO:tasks.rados.rados.0.smithi142.stdout:295: oid 46 updating version 89 to 90 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:295: oid 46 updating version 90 to 92 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:295: oid 46 version 92 is already newer than 91 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 46 v 92 (ObjNum 114 snap 26 seq_num 114) dirty exists 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:295: left oid 46 (ObjNum 114 snap 26 seq_num 114) 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:302: finishing write tid 4 to smithi14230322-4 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:302: finishing write tid 5 to smithi14230322-4 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:288: finishing write tid 5 to smithi14230322-49 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:288: oid 49 updating version 0 to 126 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:288: oid 49 updating version 126 to 129 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:288: oid 49 version 129 is already newer than 128 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:288: oid 49 version 129 is already newer than 127 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 49 v 129 (ObjNum 112 snap 26 seq_num 112) dirty exists 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:288: left oid 49 (ObjNum 112 snap 26 seq_num 112) 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:311: finishing copy_from to smithi14230322-27 2024-04-18T23:11:51.659 INFO:tasks.rados.rados.0.smithi142.stdout:311: got expected ENOENT (src dne) 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:311: finishing copy_from racing read to smithi14230322-27 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:307: finishing rollback tid 2 to smithi14230322-13 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 13 v 76 (ObjNum 94 snap 21 seq_num 94) dirty exists 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:306: finishing rollback tid 0 to smithi14230322-3 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:306: finishing rollback tid 1 to smithi14230322-3 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:309: finishing rollback tid 1 to smithi14230322-5 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:309: finishing rollback tid 2 to smithi14230322-5 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 5 v 20 (ObjNum 4 snap 0 seq_num 4) dirty exists 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:312: finishing rollback tid 0 to smithi14230322-1 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:312: finishing rollback tid 1 to smithi14230322-1 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:306: finishing rollback tid 2 to smithi14230322-3 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 3 v 108 (ObjNum 6 snap 0 seq_num 6) dirty exists 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:312: finishing rollback tid 2 to smithi14230322-1 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 1 v 6 (ObjNum 0 snap 0 seq_num 0) dirty exists 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:302: finishing write tid 6 to smithi14230322-4 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:302: oid 4 updating version 0 to 59 2024-04-18T23:11:51.660 INFO:tasks.rados.rados.0.smithi142.stdout:302: oid 4 updating version 59 to 63 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:302: oid 4 version 63 is already newer than 60 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:302: oid 4 version 63 is already newer than 61 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:302: oid 4 version 63 is already newer than 62 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 4 v 63 (ObjNum 117 snap 26 seq_num 117) dirty exists 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:302: left oid 4 (ObjNum 117 snap 26 seq_num 117) 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:305: finishing write tid 1 to smithi14230322-32 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:305: finishing write tid 2 to smithi14230322-32 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:305: finishing write tid 3 to smithi14230322-32 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:299: finishing write tid 6 to smithi14230322-38 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:299: oid 38 updating version 0 to 171 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:299: oid 38 version 171 is already newer than 169 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:299: oid 38 updating version 171 to 173 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:299: oid 38 version 173 is already newer than 172 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:299: oid 38 version 173 is already newer than 170 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 38 v 173 (ObjNum 116 snap 26 seq_num 116) dirty exists 2024-04-18T23:11:51.661 INFO:tasks.rados.rados.0.smithi142.stdout:299: left oid 38 (ObjNum 116 snap 26 seq_num 116) 2024-04-18T23:11:51.662 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-04-18T23:11:51.662 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.14 to 1 (10000) 2024-04-18T23:11:51.676 INFO:tasks.rados.rados.0.smithi142.stdout:305: finishing write tid 4 to smithi14230322-32 2024-04-18T23:11:51.682 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:51.678+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:51.692 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-04-18T23:11:51.701 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:51.701 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:51.715 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:11:51.782 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:51.778+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:51.840 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:51.840 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:51.855 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:11:51.880 INFO:tasks.rados.rados.0.smithi142.stdout:304: finishing write tid 1 to smithi14230322-44 2024-04-18T23:11:51.880 INFO:tasks.rados.rados.0.smithi142.stdout:304: finishing write tid 2 to smithi14230322-44 2024-04-18T23:11:51.880 INFO:tasks.rados.rados.0.smithi142.stdout:304: finishing write tid 3 to smithi14230322-44 2024-04-18T23:11:51.883 INFO:tasks.rados.rados.0.smithi142.stdout:305: finishing write tid 5 to smithi14230322-32 2024-04-18T23:11:51.883 INFO:tasks.rados.rados.0.smithi142.stdout:305: finishing write tid 6 to smithi14230322-32 2024-04-18T23:11:51.883 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:51.883+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:51.888 INFO:tasks.rados.rados.0.smithi142.stdout:304: finishing write tid 4 to smithi14230322-44 2024-04-18T23:11:51.888 INFO:tasks.rados.rados.0.smithi142.stdout:304: finishing write tid 5 to smithi14230322-44 2024-04-18T23:11:51.976 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:51.976 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:51.984 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:51.982+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:51.990 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:11:52.085 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:52.082+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:52.164 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:52.196 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:52.198 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:11:52.199 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:11:52.194+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:11:52.285 INFO:tasks.rados.rados.0.smithi142.stdout:305: finishing write tid 7 to smithi14230322-32 2024-04-18T23:11:52.285 INFO:tasks.rados.rados.0.smithi142.stdout:305: oid 32 updating version 0 to 177 2024-04-18T23:11:52.285 INFO:tasks.rados.rados.0.smithi142.stdout:305: oid 32 updating version 177 to 181 2024-04-18T23:11:52.285 INFO:tasks.rados.rados.0.smithi142.stdout:305: oid 32 version 181 is already newer than 178 2024-04-18T23:11:52.285 INFO:tasks.rados.rados.0.smithi142.stdout:305: oid 32 version 181 is already newer than 179 2024-04-18T23:11:52.285 INFO:tasks.rados.rados.0.smithi142.stdout:305: oid 32 updating version 181 to 182 2024-04-18T23:11:52.285 INFO:tasks.rados.rados.0.smithi142.stdout:305: oid 32 version 182 is already newer than 180 2024-04-18T23:11:52.285 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 32 v 182 (ObjNum 119 snap 26 seq_num 119) dirty exists 2024-04-18T23:11:52.285 INFO:tasks.rados.rados.0.smithi142.stdout:305: left oid 32 (ObjNum 119 snap 26 seq_num 119) 2024-04-18T23:11:52.299 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:52.294+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:52.324 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:52.325 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:52.339 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-18T23:11:52.399 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:52.394+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:52.457 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:52.457 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:52.471 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-18T23:11:52.501 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:52.498+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:52.589 INFO:tasks.rados.rados.0.smithi142.stdout:304: finishing write tid 6 to smithi14230322-44 2024-04-18T23:11:52.589 INFO:tasks.rados.rados.0.smithi142.stdout:304: oid 44 updating version 0 to 123 2024-04-18T23:11:52.589 INFO:tasks.rados.rados.0.smithi142.stdout:304: oid 44 updating version 123 to 125 2024-04-18T23:11:52.589 INFO:tasks.rados.rados.0.smithi142.stdout:304: oid 44 version 125 is already newer than 124 2024-04-18T23:11:52.589 INFO:tasks.rados.rados.0.smithi142.stdout:304: oid 44 updating version 125 to 126 2024-04-18T23:11:52.589 INFO:tasks.rados.rados.0.smithi142.stdout:304: oid 44 version 126 is already newer than 122 2024-04-18T23:11:52.589 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 44 v 126 (ObjNum 118 snap 26 seq_num 118) dirty exists 2024-04-18T23:11:52.589 INFO:tasks.rados.rados.0.smithi142.stdout:304: left oid 44 (ObjNum 118 snap 26 seq_num 118) 2024-04-18T23:11:52.593 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:52.593 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:52.602 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:52.601+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:52.607 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:11:52.657 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.15 to 1 (10000) 2024-04-18T23:11:52.658 INFO:tasks.rados.rados.0.smithi142.stdout:288: done (11 left) 2024-04-18T23:11:52.658 INFO:tasks.rados.rados.0.smithi142.stdout:295: done (10 left) 2024-04-18T23:11:52.658 INFO:tasks.rados.rados.0.smithi142.stdout:299: done (9 left) 2024-04-18T23:11:52.658 INFO:tasks.rados.rados.0.smithi142.stdout:302: done (8 left) 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:304: done (7 left) 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:305: done (6 left) 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:306: done (5 left) 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:307: done (4 left) 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:309: done (3 left) 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:311: done (2 left) 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:312: done (1 left) 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:314: done (0 left) 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 16 current snap is 27 2024-04-18T23:11:52.659 INFO:tasks.rados.rados.0.smithi142.stdout:315: seq_num 120 ranges {3932160=475136,4407296=655360,5062656=622592,5685248=278528} 2024-04-18T23:11:52.685 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-18T23:11:52.686 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:11:52.702 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:52.698+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:52.724 INFO:tasks.rados.rados.0.smithi142.stdout:315: writing smithi14230322-16 from 3932160 to 4407296 tid 1 2024-04-18T23:11:52.731 INFO:tasks.rados.rados.0.smithi142.stdout:315: writing smithi14230322-16 from 4407296 to 5062656 tid 2 2024-04-18T23:11:52.737 INFO:tasks.rados.rados.0.smithi142.stdout:315: writing smithi14230322-16 from 5062656 to 5685248 tid 3 2024-04-18T23:11:52.739 INFO:tasks.rados.rados.0.smithi142.stdout:315: writing smithi14230322-16 from 5685248 to 5963776 tid 4 2024-04-18T23:11:52.740 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 18 current snap is 27 2024-04-18T23:11:52.740 INFO:tasks.rados.rados.0.smithi142.stdout:316: seq_num 121 ranges {0=491520,491520=802816,1294336=720896,2015232=32768} 2024-04-18T23:11:52.743 INFO:tasks.rados.rados.0.smithi142.stdout:316: writing smithi14230322-18 from 0 to 491520 tid 1 2024-04-18T23:11:52.746 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:52.746 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:52.750 INFO:tasks.rados.rados.0.smithi142.stdout:316: writing smithi14230322-18 from 491520 to 1294336 tid 2 2024-04-18T23:11:52.755 INFO:tasks.rados.rados.0.smithi142.stdout:316: writing smithi14230322-18 from 1294336 to 2015232 tid 3 2024-04-18T23:11:52.755 INFO:tasks.rados.rados.0.smithi142.stdout:316: writing smithi14230322-18 from 2015232 to 2048000 tid 4 2024-04-18T23:11:52.755 INFO:tasks.rados.rados.0.smithi142.stdout:317: setattr oid 45 current snap is 27 2024-04-18T23:11:52.757 INFO:tasks.rados.rados.0.smithi142.stdout:318: snap_remove snap 11 2024-04-18T23:11:52.763 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:11:52.812 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:52.810+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:52.895 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:52.895 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:52.903 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:52.903+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:52.910 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:11:53.004 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:53.005+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:53.006 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:11:53.007 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:11:53.024 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":106,"stamp":"2024-04-18T23:11:52.655650+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":94143008,"num_objects":116,"num_object_clones":64,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":6,"num_read":530,"num_read_kb":96843,"num_write":877,"num_write_kb":107812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1252,"ondisk_log_size":1252,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":659928,"kb_used_data":176144,"kb_used_omap":35,"kb_used_meta":483548,"kb_avail":1509289512,"statfs":{"total":1546188226560,"available":1545512460288,"internally_reserved":0,"allocated":180371456,"data_stored":212920865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":495153793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,0,16,10,4,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22986752,"num_objects":12,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":-4,"num_read":42,"num_read_kb":24833,"num_write":86,"num_write_kb":22368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.147369"},"pg_stats":[{"pgid":"3.f","version":"80'19","reported_seq":271,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.765127+0000","last_change":"2024-04-18T23:11:50.766015+0000","last_active":"2024-04-18T23:11:50.765127+0000","last_peered":"2024-04-18T23:11:50.765127+0000","last_clean":"2024-04-18T23:11:50.765127+0000","last_became_active":"2024-04-18T23:10:54.696778+0000","last_became_peered":"2024-04-18T23:10:54.696778+0000","last_unstale":"2024-04-18T23:11:50.765127+0000","last_undegraded":"2024-04-18T23:11:50.765127+0000","last_fullsized":"2024-04-18T23:11:50.765127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:10.986968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013889309999999999,"stat_sum":{"num_bytes":1933312,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":13445,"num_write":6,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.e","version":"84'76","reported_seq":323,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.066056+0000","last_change":"2024-04-18T23:11:50.829585+0000","last_active":"2024-04-18T23:11:51.066056+0000","last_peered":"2024-04-18T23:11:51.066056+0000","last_clean":"2024-04-18T23:11:51.066056+0000","last_became_active":"2024-04-18T23:10:54.918517+0000","last_became_peered":"2024-04-18T23:10:54.918517+0000","last_unstale":"2024-04-18T23:11:51.066056+0000","last_undegraded":"2024-04-18T23:11:51.066056+0000","last_fullsized":"2024-04-18T23:11:51.066056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.689153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011030949,"stat_sum":{"num_bytes":6995968,"num_objects":7,"num_object_clones":4,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":16277,"num_write":54,"num_write_kb":8016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.d","version":"84'126","reported_seq":383,"reported_epoch":85,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.889965+0000","last_change":"2024-04-18T23:11:50.973051+0000","last_active":"2024-04-18T23:11:51.889965+0000","last_peered":"2024-04-18T23:11:51.889965+0000","last_clean":"2024-04-18T23:11:51.889965+0000","last_became_active":"2024-04-18T23:10:54.712663+0000","last_became_peered":"2024-04-18T23:10:54.712663+0000","last_unstale":"2024-04-18T23:11:51.889965+0000","last_undegraded":"2024-04-18T23:11:51.889965+0000","last_fullsized":"2024-04-18T23:11:51.889965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:06.509474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.81427957100000004,"stat_sum":{"num_bytes":13123584,"num_objects":14,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":50,"num_read_kb":5296,"num_write":103,"num_write_kb":15088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.c","version":"81'51","reported_seq":301,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.764896+0000","last_change":"2024-04-18T23:11:50.865568+0000","last_active":"2024-04-18T23:11:50.764896+0000","last_peered":"2024-04-18T23:11:50.764896+0000","last_clean":"2024-04-18T23:11:50.764896+0000","last_became_active":"2024-04-18T23:10:54.717053+0000","last_became_peered":"2024-04-18T23:10:54.717053+0000","last_unstale":"2024-04-18T23:11:50.764896+0000","last_undegraded":"2024-04-18T23:11:50.764896+0000","last_fullsized":"2024-04-18T23:11:50.764896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:54.955177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.101321669,"stat_sum":{"num_bytes":6815744,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":1926,"num_write":15,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.b","version":"84'133","reported_seq":381,"reported_epoch":84,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.473880+0000","last_change":"2024-04-18T23:11:50.644732+0000","last_active":"2024-04-18T23:11:51.473880+0000","last_peered":"2024-04-18T23:11:51.473880+0000","last_clean":"2024-04-18T23:11:51.473880+0000","last_became_active":"2024-04-18T23:10:54.808152+0000","last_became_peered":"2024-04-18T23:10:54.808152+0000","last_unstale":"2024-04-18T23:11:51.473880+0000","last_undegraded":"2024-04-18T23:11:51.473880+0000","last_fullsized":"2024-04-18T23:11:51.473880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:59.990562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.51132370100000002,"stat_sum":{"num_bytes":13631488,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":33,"num_read_kb":8320,"num_write":45,"num_write_kb":13584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.a","version":"84'20","reported_seq":280,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.227940+0000","last_change":"2024-04-18T23:11:50.820320+0000","last_active":"2024-04-18T23:11:51.227940+0000","last_peered":"2024-04-18T23:11:51.227940+0000","last_clean":"2024-04-18T23:11:51.227940+0000","last_became_active":"2024-04-18T23:10:54.764999+0000","last_became_peered":"2024-04-18T23:10:54.764999+0000","last_unstale":"2024-04-18T23:11:51.227940+0000","last_undegraded":"2024-04-18T23:11:51.227940+0000","last_fullsized":"2024-04-18T23:11:51.227940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:00.989345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10087621099999999,"stat_sum":{"num_bytes":720896,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":3750,"num_write":6,"num_write_kb":624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.9","version":"84'108","reported_seq":407,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.190183+0000","last_change":"2024-04-18T23:11:50.988961+0000","last_active":"2024-04-18T23:11:51.190183+0000","last_peered":"2024-04-18T23:11:51.190183+0000","last_clean":"2024-04-18T23:11:51.190183+0000","last_became_active":"2024-04-18T23:11:39.726824+0000","last_became_peered":"2024-04-18T23:11:39.726824+0000","last_unstale":"2024-04-18T23:11:51.190183+0000","last_undegraded":"2024-04-18T23:11:51.190183+0000","last_fullsized":"2024-04-18T23:11:51.190183+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:02.111340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21817656799999999,"stat_sum":{"num_bytes":5816320,"num_objects":14,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":55,"num_read_kb":10769,"num_write":81,"num_write_kb":7696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2867200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.8","version":"84'55","reported_seq":331,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.869640+0000","last_change":"2024-04-18T23:11:50.871103+0000","last_active":"2024-04-18T23:11:50.869640+0000","last_peered":"2024-04-18T23:11:50.869640+0000","last_clean":"2024-04-18T23:11:50.869640+0000","last_became_active":"2024-04-18T23:11:37.876250+0000","last_became_peered":"2024-04-18T23:11:37.876250+0000","last_unstale":"2024-04-18T23:11:50.869640+0000","last_undegraded":"2024-04-18T23:11:50.869640+0000","last_fullsized":"2024-04-18T23:11:50.869640+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:17.115381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20782552300000001,"stat_sum":{"num_bytes":2129920,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":2250,"num_write":54,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"1.0","version":"33'157","reported_seq":336,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.647191+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:11:50.647191+0000","last_peered":"2024-04-18T23:11:50.647191+0000","last_clean":"2024-04-18T23:11:50.647191+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:11:50.647191+0000","last_undegraded":"2024-04-18T23:11:50.647191+0000","last_fullsized":"2024-04-18T23:11:50.647191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.763430+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:11:50.763430+0000","last_peered":"2024-04-18T23:11:50.763430+0000","last_clean":"2024-04-18T23:11:50.763430+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:11:50.763430+0000","last_undegraded":"2024-04-18T23:11:50.763430+0000","last_fullsized":"2024-04-18T23:11:50.763430+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"84'65","reported_seq":338,"reported_epoch":84,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:50.982419+0000","last_change":"2024-04-18T23:11:50.881001+0000","last_active":"2024-04-18T23:11:50.982419+0000","last_peered":"2024-04-18T23:11:50.982419+0000","last_clean":"2024-04-18T23:11:50.982419+0000","last_became_active":"2024-04-18T23:11:33.818951+0000","last_became_peered":"2024-04-18T23:11:33.818951+0000","last_unstale":"2024-04-18T23:11:50.982419+0000","last_undegraded":"2024-04-18T23:11:50.982419+0000","last_fullsized":"2024-04-18T23:11:50.982419+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:22.058860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.523503465,"stat_sum":{"num_bytes":9797632,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":6012,"num_write":36,"num_write_kb":9872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":126,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.819088+0000","last_change":"2024-04-18T23:10:45.497972+0000","last_active":"2024-04-18T23:11:50.819088+0000","last_peered":"2024-04-18T23:11:50.819088+0000","last_clean":"2024-04-18T23:11:50.819088+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:11:50.819088+0000","last_undegraded":"2024-04-18T23:11:50.819088+0000","last_fullsized":"2024-04-18T23:11:50.819088+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.071639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"84'6","reported_seq":251,"reported_epoch":84,"state":"active+clean+snaptrim_wait","last_fresh":"2024-04-18T23:11:51.271218+0000","last_change":"2024-04-18T23:11:50.744771+0000","last_active":"2024-04-18T23:11:51.271218+0000","last_peered":"2024-04-18T23:11:51.271218+0000","last_clean":"2024-04-18T23:11:51.271218+0000","last_became_active":"2024-04-18T23:10:54.799189+0000","last_became_peered":"2024-04-18T23:10:54.799189+0000","last_unstale":"2024-04-18T23:11:51.271218+0000","last_undegraded":"2024-04-18T23:11:51.271218+0000","last_fullsized":"2024-04-18T23:11:51.271218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.458672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082261200000000002,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.647289+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:11:50.647289+0000","last_peered":"2024-04-18T23:11:50.647289+0000","last_clean":"2024-04-18T23:11:50.647289+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:11:50.647289+0000","last_undegraded":"2024-04-18T23:11:50.647289+0000","last_fullsized":"2024-04-18T23:11:50.647289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:14.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"81'34","reported_seq":314,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.670683+0000","last_change":"2024-04-18T23:11:50.771452+0000","last_active":"2024-04-18T23:11:50.670683+0000","last_peered":"2024-04-18T23:11:50.670683+0000","last_clean":"2024-04-18T23:11:50.670683+0000","last_became_active":"2024-04-18T23:11:38.818022+0000","last_became_peered":"2024-04-18T23:11:38.818022+0000","last_unstale":"2024-04-18T23:11:50.670683+0000","last_undegraded":"2024-04-18T23:11:50.670683+0000","last_fullsized":"2024-04-18T23:11:50.670683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:17.808734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10106219900000001,"stat_sum":{"num_bytes":4407296,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2791,"num_write":15,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":7716864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":125,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.819341+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:11:50.819341+0000","last_peered":"2024-04-18T23:11:50.819341+0000","last_clean":"2024-04-18T23:11:50.819341+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:11:50.819341+0000","last_undegraded":"2024-04-18T23:11:50.819341+0000","last_fullsized":"2024-04-18T23:11:50.819341+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.994274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":260,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.647954+0000","last_change":"2024-04-18T23:11:50.648670+0000","last_active":"2024-04-18T23:11:50.647954+0000","last_peered":"2024-04-18T23:11:50.647954+0000","last_clean":"2024-04-18T23:11:50.647954+0000","last_became_active":"2024-04-18T23:11:33.713370+0000","last_became_peered":"2024-04-18T23:11:33.713370+0000","last_unstale":"2024-04-18T23:11:50.647954+0000","last_undegraded":"2024-04-18T23:11:50.647954+0000","last_fullsized":"2024-04-18T23:11:50.647954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:04.176081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062739,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.5","version":"84'98","reported_seq":375,"reported_epoch":85,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.679200+0000","last_change":"2024-04-18T23:11:51.679241+0000","last_active":"2024-04-18T23:11:51.679200+0000","last_peered":"2024-04-18T23:11:51.679200+0000","last_clean":"2024-04-18T23:11:51.679200+0000","last_became_active":"2024-04-18T23:11:39.923911+0000","last_became_peered":"2024-04-18T23:11:39.923911+0000","last_unstale":"2024-04-18T23:11:51.679200+0000","last_undegraded":"2024-04-18T23:11:51.679200+0000","last_fullsized":"2024-04-18T23:11:51.679200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.743945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.0097257239999999,"stat_sum":{"num_bytes":7716864,"num_objects":11,"num_object_clones":8,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":39,"num_read_kb":12301,"num_write":51,"num_write_kb":9744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5685248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.644455+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:11:50.644455+0000","last_peered":"2024-04-18T23:11:50.644455+0000","last_clean":"2024-04-18T23:11:50.644455+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:11:50.644455+0000","last_undegraded":"2024-04-18T23:11:50.644455+0000","last_fullsized":"2024-04-18T23:11:50.644455+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"84'82","reported_seq":322,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.982890+0000","last_change":"2024-04-18T23:11:50.983789+0000","last_active":"2024-04-18T23:11:50.982890+0000","last_peered":"2024-04-18T23:11:50.982890+0000","last_clean":"2024-04-18T23:11:50.982890+0000","last_became_active":"2024-04-18T23:10:54.793319+0000","last_became_peered":"2024-04-18T23:10:54.793319+0000","last_unstale":"2024-04-18T23:11:50.982890+0000","last_undegraded":"2024-04-18T23:11:50.982890+0000","last_fullsized":"2024-04-18T23:11:50.982890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:11.752239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30504247200000001,"stat_sum":{"num_bytes":3063808,"num_objects":9,"num_object_clones":5,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":14,"num_read_kb":6,"num_write":23,"num_write_kb":3408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":126,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.661941+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:11:50.661941+0000","last_peered":"2024-04-18T23:11:50.661941+0000","last_clean":"2024-04-18T23:11:50.661941+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:11:50.661941+0000","last_undegraded":"2024-04-18T23:11:50.661941+0000","last_fullsized":"2024-04-18T23:11:50.661941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.503236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"84'182","reported_seq":425,"reported_epoch":84,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.476470+0000","last_change":"2024-04-18T23:11:50.657733+0000","last_active":"2024-04-18T23:11:51.476470+0000","last_peered":"2024-04-18T23:11:51.476470+0000","last_clean":"2024-04-18T23:11:51.476470+0000","last_became_active":"2024-04-18T23:10:54.693609+0000","last_became_peered":"2024-04-18T23:10:54.693609+0000","last_unstale":"2024-04-18T23:11:51.476470+0000","last_undegraded":"2024-04-18T23:11:51.476470+0000","last_fullsized":"2024-04-18T23:11:51.476470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:13.905711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.472976709,"stat_sum":{"num_bytes":13500416,"num_objects":17,"num_object_clones":9,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":47,"num_read_kb":11702,"num_write":163,"num_write_kb":14880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.678768+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:11:50.678768+0000","last_peered":"2024-04-18T23:11:50.678768+0000","last_clean":"2024-04-18T23:11:50.678768+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:11:50.678768+0000","last_undegraded":"2024-04-18T23:11:50.678768+0000","last_fullsized":"2024-04-18T23:11:50.678768+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.869947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"83'40","reported_seq":321,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.660532+0000","last_change":"2024-04-18T23:11:50.761229+0000","last_active":"2024-04-18T23:11:50.660532+0000","last_peered":"2024-04-18T23:11:50.660532+0000","last_clean":"2024-04-18T23:11:50.660532+0000","last_became_active":"2024-04-18T23:11:39.717443+0000","last_became_peered":"2024-04-18T23:11:39.717443+0000","last_unstale":"2024-04-18T23:11:50.660532+0000","last_undegraded":"2024-04-18T23:11:50.660532+0000","last_fullsized":"2024-04-18T23:11:50.660532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:00.941867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.100906204,"stat_sum":{"num_bytes":3866624,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":1879,"num_write":18,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.662204+0000","last_change":"2024-04-18T23:11:33.519671+0000","last_active":"2024-04-18T23:11:51.662204+0000","last_peered":"2024-04-18T23:11:51.662204+0000","last_clean":"2024-04-18T23:11:51.662204+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:11:51.662204+0000","last_undegraded":"2024-04-18T23:11:51.662204+0000","last_fullsized":"2024-04-18T23:11:51.662204+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":93552640,"num_objects":114,"num_object_clones":64,"num_object_copies":798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":6,"num_read":394,"num_read_kb":96725,"num_write":674,"num_write_kb":104352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163749888,"data_stored":211496960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0},"log_size":1095,"ondisk_log_size":1095,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808837,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38580,"kb_used_data":10016,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94333260,"statfs":{"total":96636764160,"available":96597258240,"internally_reserved":0,"allocated":10256384,"data_stored":11702167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,1,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39968,"kb_used_data":8784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331872,"statfs":{"total":96636764160,"available":96595836928,"internally_reserved":0,"allocated":8994816,"data_stored":9893638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280530,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39336,"kb_used_data":8156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332504,"statfs":{"total":96636764160,"available":96596484096,"internally_reserved":0,"allocated":8351744,"data_stored":9636134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313233,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49716,"kb_used_data":18536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322124,"statfs":{"total":96636764160,"available":96585854976,"internally_reserved":0,"allocated":18980864,"data_stored":22669062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313233,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44660,"kb_used_data":13480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327180,"statfs":{"total":96636764160,"available":96591032320,"internally_reserved":0,"allocated":13803520,"data_stored":16643846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313234,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34872,"kb_used_data":7856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336968,"statfs":{"total":96636764160,"available":96601055232,"internally_reserved":0,"allocated":8044544,"data_stored":9104979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313234,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45404,"kb_used_data":14224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326436,"statfs":{"total":96636764160,"available":96590270464,"internally_reserved":0,"allocated":14565376,"data_stored":17917702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313234,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33976,"kb_used_data":6960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337864,"statfs":{"total":96636764160,"available":96601972736,"internally_reserved":0,"allocated":7127040,"data_stored":7376467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,7,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313232,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45784,"kb_used_data":14604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326056,"statfs":{"total":96636764160,"available":96589881344,"internally_reserved":0,"allocated":14954496,"data_stored":18700038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313232,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":10376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":10625024,"data_stored":11779667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313233,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39996,"kb_used_data":8816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331844,"statfs":{"total":96636764160,"available":96595808256,"internally_reserved":0,"allocated":9027584,"data_stored":10489427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280530,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39792,"kb_used_data":8616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332048,"statfs":{"total":96636764160,"available":96596017152,"internally_reserved":0,"allocated":8822784,"data_stored":9977971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280530,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47596,"kb_used_data":16420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324244,"statfs":{"total":96636764160,"available":96588025856,"internally_reserved":0,"allocated":16814080,"data_stored":20958803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,4,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280531,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41904,"kb_used_data":10720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329936,"statfs":{"total":96636764160,"available":96593854464,"internally_reserved":0,"allocated":10977280,"data_stored":13291415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280530,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41792,"kb_used_data":10608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330048,"statfs":{"total":96636764160,"available":96593969152,"internally_reserved":0,"allocated":10862592,"data_stored":14059270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280530,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39160,"kb_used_data":7972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332680,"statfs":{"total":96636764160,"available":96596664320,"internally_reserved":0,"allocated":8163328,"data_stored":8720279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17879040,"data_stored":22052864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":17158144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7090176,"data_stored":9932800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7344128,"data_stored":8962048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13869056,"data_stored":17727488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6701056,"data_stored":7847936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16388096,"data_stored":22294528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9433088,"data_stored":12075008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":14721024,"data_stored":19382272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9666560,"data_stored":13377536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":8007680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":9908224,"data_stored":13148160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":7827456,"data_stored":10964992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":6602752,"data_stored":8413184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7778304,"data_stored":9162752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":9023488,"data_stored":10989568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0}]}} 2024-04-18T23:11:53.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:11:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:53.069 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:53.085 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:11:53.105 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:53.103+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:53.206 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:53.202+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:53.238 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:53.251 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:11:53.307 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:53.306+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:53.315 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:11:53.316 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:11:53.333 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-18T23:11:53.285394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":94143008,"num_objects":116,"num_object_clones":64,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":6,"num_read":530,"num_read_kb":96843,"num_write":877,"num_write_kb":107812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1252,"ondisk_log_size":1252,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":659928,"kb_used_data":176144,"kb_used_omap":35,"kb_used_meta":483548,"kb_avail":1509289512,"statfs":{"total":1546188226560,"available":1545512460288,"internally_reserved":0,"allocated":180371456,"data_stored":212920865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":495153793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,0,16,10,4,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22986752,"num_objects":12,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":-4,"num_read":42,"num_read_kb":24833,"num_write":86,"num_write_kb":22368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001187"},"pg_stats":[{"pgid":"3.f","version":"80'19","reported_seq":271,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.765127+0000","last_change":"2024-04-18T23:11:50.766015+0000","last_active":"2024-04-18T23:11:50.765127+0000","last_peered":"2024-04-18T23:11:50.765127+0000","last_clean":"2024-04-18T23:11:50.765127+0000","last_became_active":"2024-04-18T23:10:54.696778+0000","last_became_peered":"2024-04-18T23:10:54.696778+0000","last_unstale":"2024-04-18T23:11:50.765127+0000","last_undegraded":"2024-04-18T23:11:50.765127+0000","last_fullsized":"2024-04-18T23:11:50.765127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:10.986968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013889309999999999,"stat_sum":{"num_bytes":1933312,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":13445,"num_write":6,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.e","version":"84'76","reported_seq":323,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.066056+0000","last_change":"2024-04-18T23:11:50.829585+0000","last_active":"2024-04-18T23:11:51.066056+0000","last_peered":"2024-04-18T23:11:51.066056+0000","last_clean":"2024-04-18T23:11:51.066056+0000","last_became_active":"2024-04-18T23:10:54.918517+0000","last_became_peered":"2024-04-18T23:10:54.918517+0000","last_unstale":"2024-04-18T23:11:51.066056+0000","last_undegraded":"2024-04-18T23:11:51.066056+0000","last_fullsized":"2024-04-18T23:11:51.066056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.689153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011030949,"stat_sum":{"num_bytes":6995968,"num_objects":7,"num_object_clones":4,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":16277,"num_write":54,"num_write_kb":8016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.d","version":"84'126","reported_seq":383,"reported_epoch":85,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.889965+0000","last_change":"2024-04-18T23:11:50.973051+0000","last_active":"2024-04-18T23:11:51.889965+0000","last_peered":"2024-04-18T23:11:51.889965+0000","last_clean":"2024-04-18T23:11:51.889965+0000","last_became_active":"2024-04-18T23:10:54.712663+0000","last_became_peered":"2024-04-18T23:10:54.712663+0000","last_unstale":"2024-04-18T23:11:51.889965+0000","last_undegraded":"2024-04-18T23:11:51.889965+0000","last_fullsized":"2024-04-18T23:11:51.889965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:06.509474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.81427957100000004,"stat_sum":{"num_bytes":13123584,"num_objects":14,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":50,"num_read_kb":5296,"num_write":103,"num_write_kb":15088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.c","version":"81'51","reported_seq":301,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.764896+0000","last_change":"2024-04-18T23:11:50.865568+0000","last_active":"2024-04-18T23:11:50.764896+0000","last_peered":"2024-04-18T23:11:50.764896+0000","last_clean":"2024-04-18T23:11:50.764896+0000","last_became_active":"2024-04-18T23:10:54.717053+0000","last_became_peered":"2024-04-18T23:10:54.717053+0000","last_unstale":"2024-04-18T23:11:50.764896+0000","last_undegraded":"2024-04-18T23:11:50.764896+0000","last_fullsized":"2024-04-18T23:11:50.764896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:54.955177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.101321669,"stat_sum":{"num_bytes":6815744,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":1926,"num_write":15,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.b","version":"84'133","reported_seq":381,"reported_epoch":84,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.473880+0000","last_change":"2024-04-18T23:11:50.644732+0000","last_active":"2024-04-18T23:11:51.473880+0000","last_peered":"2024-04-18T23:11:51.473880+0000","last_clean":"2024-04-18T23:11:51.473880+0000","last_became_active":"2024-04-18T23:10:54.808152+0000","last_became_peered":"2024-04-18T23:10:54.808152+0000","last_unstale":"2024-04-18T23:11:51.473880+0000","last_undegraded":"2024-04-18T23:11:51.473880+0000","last_fullsized":"2024-04-18T23:11:51.473880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:59.990562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.51132370100000002,"stat_sum":{"num_bytes":13631488,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":33,"num_read_kb":8320,"num_write":45,"num_write_kb":13584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.a","version":"84'20","reported_seq":280,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.227940+0000","last_change":"2024-04-18T23:11:50.820320+0000","last_active":"2024-04-18T23:11:51.227940+0000","last_peered":"2024-04-18T23:11:51.227940+0000","last_clean":"2024-04-18T23:11:51.227940+0000","last_became_active":"2024-04-18T23:10:54.764999+0000","last_became_peered":"2024-04-18T23:10:54.764999+0000","last_unstale":"2024-04-18T23:11:51.227940+0000","last_undegraded":"2024-04-18T23:11:51.227940+0000","last_fullsized":"2024-04-18T23:11:51.227940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:00.989345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10087621099999999,"stat_sum":{"num_bytes":720896,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":3750,"num_write":6,"num_write_kb":624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.9","version":"84'108","reported_seq":407,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.190183+0000","last_change":"2024-04-18T23:11:50.988961+0000","last_active":"2024-04-18T23:11:51.190183+0000","last_peered":"2024-04-18T23:11:51.190183+0000","last_clean":"2024-04-18T23:11:51.190183+0000","last_became_active":"2024-04-18T23:11:39.726824+0000","last_became_peered":"2024-04-18T23:11:39.726824+0000","last_unstale":"2024-04-18T23:11:51.190183+0000","last_undegraded":"2024-04-18T23:11:51.190183+0000","last_fullsized":"2024-04-18T23:11:51.190183+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:02.111340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21817656799999999,"stat_sum":{"num_bytes":5816320,"num_objects":14,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":55,"num_read_kb":10769,"num_write":81,"num_write_kb":7696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2867200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.8","version":"84'55","reported_seq":331,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.869640+0000","last_change":"2024-04-18T23:11:50.871103+0000","last_active":"2024-04-18T23:11:50.869640+0000","last_peered":"2024-04-18T23:11:50.869640+0000","last_clean":"2024-04-18T23:11:50.869640+0000","last_became_active":"2024-04-18T23:11:37.876250+0000","last_became_peered":"2024-04-18T23:11:37.876250+0000","last_unstale":"2024-04-18T23:11:50.869640+0000","last_undegraded":"2024-04-18T23:11:50.869640+0000","last_fullsized":"2024-04-18T23:11:50.869640+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:17.115381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20782552300000001,"stat_sum":{"num_bytes":2129920,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":2250,"num_write":54,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"1.0","version":"33'157","reported_seq":336,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.647191+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:11:50.647191+0000","last_peered":"2024-04-18T23:11:50.647191+0000","last_clean":"2024-04-18T23:11:50.647191+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:11:50.647191+0000","last_undegraded":"2024-04-18T23:11:50.647191+0000","last_fullsized":"2024-04-18T23:11:50.647191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.763430+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:11:50.763430+0000","last_peered":"2024-04-18T23:11:50.763430+0000","last_clean":"2024-04-18T23:11:50.763430+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:11:50.763430+0000","last_undegraded":"2024-04-18T23:11:50.763430+0000","last_fullsized":"2024-04-18T23:11:50.763430+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"84'65","reported_seq":338,"reported_epoch":84,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:50.982419+0000","last_change":"2024-04-18T23:11:50.881001+0000","last_active":"2024-04-18T23:11:50.982419+0000","last_peered":"2024-04-18T23:11:50.982419+0000","last_clean":"2024-04-18T23:11:50.982419+0000","last_became_active":"2024-04-18T23:11:33.818951+0000","last_became_peered":"2024-04-18T23:11:33.818951+0000","last_unstale":"2024-04-18T23:11:50.982419+0000","last_undegraded":"2024-04-18T23:11:50.982419+0000","last_fullsized":"2024-04-18T23:11:50.982419+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:22.058860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.523503465,"stat_sum":{"num_bytes":9797632,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":6012,"num_write":36,"num_write_kb":9872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":126,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.819088+0000","last_change":"2024-04-18T23:10:45.497972+0000","last_active":"2024-04-18T23:11:50.819088+0000","last_peered":"2024-04-18T23:11:50.819088+0000","last_clean":"2024-04-18T23:11:50.819088+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:11:50.819088+0000","last_undegraded":"2024-04-18T23:11:50.819088+0000","last_fullsized":"2024-04-18T23:11:50.819088+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.071639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"84'6","reported_seq":251,"reported_epoch":84,"state":"active+clean+snaptrim_wait","last_fresh":"2024-04-18T23:11:51.271218+0000","last_change":"2024-04-18T23:11:50.744771+0000","last_active":"2024-04-18T23:11:51.271218+0000","last_peered":"2024-04-18T23:11:51.271218+0000","last_clean":"2024-04-18T23:11:51.271218+0000","last_became_active":"2024-04-18T23:10:54.799189+0000","last_became_peered":"2024-04-18T23:10:54.799189+0000","last_unstale":"2024-04-18T23:11:51.271218+0000","last_undegraded":"2024-04-18T23:11:51.271218+0000","last_fullsized":"2024-04-18T23:11:51.271218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.458672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082261200000000002,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.647289+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:11:50.647289+0000","last_peered":"2024-04-18T23:11:50.647289+0000","last_clean":"2024-04-18T23:11:50.647289+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:11:50.647289+0000","last_undegraded":"2024-04-18T23:11:50.647289+0000","last_fullsized":"2024-04-18T23:11:50.647289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:14.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"81'34","reported_seq":314,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.670683+0000","last_change":"2024-04-18T23:11:50.771452+0000","last_active":"2024-04-18T23:11:50.670683+0000","last_peered":"2024-04-18T23:11:50.670683+0000","last_clean":"2024-04-18T23:11:50.670683+0000","last_became_active":"2024-04-18T23:11:38.818022+0000","last_became_peered":"2024-04-18T23:11:38.818022+0000","last_unstale":"2024-04-18T23:11:50.670683+0000","last_undegraded":"2024-04-18T23:11:50.670683+0000","last_fullsized":"2024-04-18T23:11:50.670683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:17.808734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10106219900000001,"stat_sum":{"num_bytes":4407296,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2791,"num_write":15,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":7716864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":125,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.819341+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:11:50.819341+0000","last_peered":"2024-04-18T23:11:50.819341+0000","last_clean":"2024-04-18T23:11:50.819341+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:11:50.819341+0000","last_undegraded":"2024-04-18T23:11:50.819341+0000","last_fullsized":"2024-04-18T23:11:50.819341+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.994274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":260,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.647954+0000","last_change":"2024-04-18T23:11:50.648670+0000","last_active":"2024-04-18T23:11:50.647954+0000","last_peered":"2024-04-18T23:11:50.647954+0000","last_clean":"2024-04-18T23:11:50.647954+0000","last_became_active":"2024-04-18T23:11:33.713370+0000","last_became_peered":"2024-04-18T23:11:33.713370+0000","last_unstale":"2024-04-18T23:11:50.647954+0000","last_undegraded":"2024-04-18T23:11:50.647954+0000","last_fullsized":"2024-04-18T23:11:50.647954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:04.176081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062739,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.5","version":"84'98","reported_seq":375,"reported_epoch":85,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.679200+0000","last_change":"2024-04-18T23:11:51.679241+0000","last_active":"2024-04-18T23:11:51.679200+0000","last_peered":"2024-04-18T23:11:51.679200+0000","last_clean":"2024-04-18T23:11:51.679200+0000","last_became_active":"2024-04-18T23:11:39.923911+0000","last_became_peered":"2024-04-18T23:11:39.923911+0000","last_unstale":"2024-04-18T23:11:51.679200+0000","last_undegraded":"2024-04-18T23:11:51.679200+0000","last_fullsized":"2024-04-18T23:11:51.679200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.743945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.0097257239999999,"stat_sum":{"num_bytes":7716864,"num_objects":11,"num_object_clones":8,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":39,"num_read_kb":12301,"num_write":51,"num_write_kb":9744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5685248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.644455+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:11:50.644455+0000","last_peered":"2024-04-18T23:11:50.644455+0000","last_clean":"2024-04-18T23:11:50.644455+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:11:50.644455+0000","last_undegraded":"2024-04-18T23:11:50.644455+0000","last_fullsized":"2024-04-18T23:11:50.644455+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"84'82","reported_seq":322,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.982890+0000","last_change":"2024-04-18T23:11:50.983789+0000","last_active":"2024-04-18T23:11:50.982890+0000","last_peered":"2024-04-18T23:11:50.982890+0000","last_clean":"2024-04-18T23:11:50.982890+0000","last_became_active":"2024-04-18T23:10:54.793319+0000","last_became_peered":"2024-04-18T23:10:54.793319+0000","last_unstale":"2024-04-18T23:11:50.982890+0000","last_undegraded":"2024-04-18T23:11:50.982890+0000","last_fullsized":"2024-04-18T23:11:50.982890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:11.752239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30504247200000001,"stat_sum":{"num_bytes":3063808,"num_objects":9,"num_object_clones":5,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":14,"num_read_kb":6,"num_write":23,"num_write_kb":3408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":126,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.661941+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:11:50.661941+0000","last_peered":"2024-04-18T23:11:50.661941+0000","last_clean":"2024-04-18T23:11:50.661941+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:11:50.661941+0000","last_undegraded":"2024-04-18T23:11:50.661941+0000","last_fullsized":"2024-04-18T23:11:50.661941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.503236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"84'182","reported_seq":425,"reported_epoch":84,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.476470+0000","last_change":"2024-04-18T23:11:50.657733+0000","last_active":"2024-04-18T23:11:51.476470+0000","last_peered":"2024-04-18T23:11:51.476470+0000","last_clean":"2024-04-18T23:11:51.476470+0000","last_became_active":"2024-04-18T23:10:54.693609+0000","last_became_peered":"2024-04-18T23:10:54.693609+0000","last_unstale":"2024-04-18T23:11:51.476470+0000","last_undegraded":"2024-04-18T23:11:51.476470+0000","last_fullsized":"2024-04-18T23:11:51.476470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:13.905711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.472976709,"stat_sum":{"num_bytes":13500416,"num_objects":17,"num_object_clones":9,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":47,"num_read_kb":11702,"num_write":163,"num_write_kb":14880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.678768+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:11:50.678768+0000","last_peered":"2024-04-18T23:11:50.678768+0000","last_clean":"2024-04-18T23:11:50.678768+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:11:50.678768+0000","last_undegraded":"2024-04-18T23:11:50.678768+0000","last_fullsized":"2024-04-18T23:11:50.678768+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.869947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"83'40","reported_seq":321,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.660532+0000","last_change":"2024-04-18T23:11:50.761229+0000","last_active":"2024-04-18T23:11:50.660532+0000","last_peered":"2024-04-18T23:11:50.660532+0000","last_clean":"2024-04-18T23:11:50.660532+0000","last_became_active":"2024-04-18T23:11:39.717443+0000","last_became_peered":"2024-04-18T23:11:39.717443+0000","last_unstale":"2024-04-18T23:11:50.660532+0000","last_undegraded":"2024-04-18T23:11:50.660532+0000","last_fullsized":"2024-04-18T23:11:50.660532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:00.941867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.100906204,"stat_sum":{"num_bytes":3866624,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":1879,"num_write":18,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.662204+0000","last_change":"2024-04-18T23:11:33.519671+0000","last_active":"2024-04-18T23:11:51.662204+0000","last_peered":"2024-04-18T23:11:51.662204+0000","last_clean":"2024-04-18T23:11:51.662204+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:11:51.662204+0000","last_undegraded":"2024-04-18T23:11:51.662204+0000","last_fullsized":"2024-04-18T23:11:51.662204+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":93552640,"num_objects":114,"num_object_clones":64,"num_object_copies":798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":6,"num_read":394,"num_read_kb":96725,"num_write":674,"num_write_kb":104352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163749888,"data_stored":211496960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0},"log_size":1095,"ondisk_log_size":1095,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808837,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38580,"kb_used_data":10016,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94333260,"statfs":{"total":96636764160,"available":96597258240,"internally_reserved":0,"allocated":10256384,"data_stored":11702167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,1,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39968,"kb_used_data":8784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331872,"statfs":{"total":96636764160,"available":96595836928,"internally_reserved":0,"allocated":8994816,"data_stored":9893638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280530,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39336,"kb_used_data":8156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332504,"statfs":{"total":96636764160,"available":96596484096,"internally_reserved":0,"allocated":8351744,"data_stored":9636134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313233,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49716,"kb_used_data":18536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322124,"statfs":{"total":96636764160,"available":96585854976,"internally_reserved":0,"allocated":18980864,"data_stored":22669062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313233,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44660,"kb_used_data":13480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327180,"statfs":{"total":96636764160,"available":96591032320,"internally_reserved":0,"allocated":13803520,"data_stored":16643846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313234,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34872,"kb_used_data":7856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336968,"statfs":{"total":96636764160,"available":96601055232,"internally_reserved":0,"allocated":8044544,"data_stored":9104979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313234,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45404,"kb_used_data":14224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326436,"statfs":{"total":96636764160,"available":96590270464,"internally_reserved":0,"allocated":14565376,"data_stored":17917702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313234,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33976,"kb_used_data":6960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337864,"statfs":{"total":96636764160,"available":96601972736,"internally_reserved":0,"allocated":7127040,"data_stored":7376467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,7,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313232,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45784,"kb_used_data":14604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326056,"statfs":{"total":96636764160,"available":96589881344,"internally_reserved":0,"allocated":14954496,"data_stored":18700038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313232,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":10376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":10625024,"data_stored":11779667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313233,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39996,"kb_used_data":8816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331844,"statfs":{"total":96636764160,"available":96595808256,"internally_reserved":0,"allocated":9027584,"data_stored":10489427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280530,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39792,"kb_used_data":8616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332048,"statfs":{"total":96636764160,"available":96596017152,"internally_reserved":0,"allocated":8822784,"data_stored":9977971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280530,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47596,"kb_used_data":16420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324244,"statfs":{"total":96636764160,"available":96588025856,"internally_reserved":0,"allocated":16814080,"data_stored":20958803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,4,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280531,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41904,"kb_used_data":10720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329936,"statfs":{"total":96636764160,"available":96593854464,"internally_reserved":0,"allocated":10977280,"data_stored":13291415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280530,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41792,"kb_used_data":10608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330048,"statfs":{"total":96636764160,"available":96593969152,"internally_reserved":0,"allocated":10862592,"data_stored":14059270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280530,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39160,"kb_used_data":7972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332680,"statfs":{"total":96636764160,"available":96596664320,"internally_reserved":0,"allocated":8163328,"data_stored":8720279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17879040,"data_stored":22052864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":17158144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7090176,"data_stored":9932800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7344128,"data_stored":8962048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13869056,"data_stored":17727488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6701056,"data_stored":7847936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16388096,"data_stored":22294528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9433088,"data_stored":12075008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":14721024,"data_stored":19382272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9666560,"data_stored":13377536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":8007680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":9908224,"data_stored":13148160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":7827456,"data_stored":10964992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":6602752,"data_stored":8413184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7778304,"data_stored":9162752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":9023488,"data_stored":10989568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0}]}} 2024-04-18T23:11:53.334 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-18T23:11:53.334 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-18T23:11:53.335 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-04-18T23:11:53.335 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 16 2024-04-18T23:11:53.335 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:11:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:53.368 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:53.382 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:11:53.407 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:53.409+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:53.509 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:53.506+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:53.517 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:53.542 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:53.542 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:53.556 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:11:53.609 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:53.609+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:53.619 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:11:53.619 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:11:53.637 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":107,"stamp":"2024-04-18T23:11:53.285394+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":94143008,"num_objects":116,"num_object_clones":64,"num_object_copies":802,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":6,"num_read":530,"num_read_kb":96843,"num_write":877,"num_write_kb":107812,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1252,"ondisk_log_size":1252,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":659928,"kb_used_data":176144,"kb_used_omap":35,"kb_used_meta":483548,"kb_avail":1509289512,"statfs":{"total":1546188226560,"available":1545512460288,"internally_reserved":0,"allocated":180371456,"data_stored":212920865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":495153793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1,0,0,0,0,16,10,4,0,2],"upper_bound":2048},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22986752,"num_objects":12,"num_object_clones":11,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":-4,"num_read":42,"num_read_kb":24833,"num_write":86,"num_write_kb":22368,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001187"},"pg_stats":[{"pgid":"3.f","version":"80'19","reported_seq":271,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.765127+0000","last_change":"2024-04-18T23:11:50.766015+0000","last_active":"2024-04-18T23:11:50.765127+0000","last_peered":"2024-04-18T23:11:50.765127+0000","last_clean":"2024-04-18T23:11:50.765127+0000","last_became_active":"2024-04-18T23:10:54.696778+0000","last_became_peered":"2024-04-18T23:10:54.696778+0000","last_unstale":"2024-04-18T23:11:50.765127+0000","last_undegraded":"2024-04-18T23:11:50.765127+0000","last_fullsized":"2024-04-18T23:11:50.765127+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:10.986968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013889309999999999,"stat_sum":{"num_bytes":1933312,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":13445,"num_write":6,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.e","version":"84'76","reported_seq":323,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.066056+0000","last_change":"2024-04-18T23:11:50.829585+0000","last_active":"2024-04-18T23:11:51.066056+0000","last_peered":"2024-04-18T23:11:51.066056+0000","last_clean":"2024-04-18T23:11:51.066056+0000","last_became_active":"2024-04-18T23:10:54.918517+0000","last_became_peered":"2024-04-18T23:10:54.918517+0000","last_unstale":"2024-04-18T23:11:51.066056+0000","last_undegraded":"2024-04-18T23:11:51.066056+0000","last_fullsized":"2024-04-18T23:11:51.066056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":76,"log_dups_size":0,"ondisk_log_size":76,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.689153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011030949,"stat_sum":{"num_bytes":6995968,"num_objects":7,"num_object_clones":4,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":23,"num_read_kb":16277,"num_write":54,"num_write_kb":8016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.d","version":"84'126","reported_seq":383,"reported_epoch":85,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.889965+0000","last_change":"2024-04-18T23:11:50.973051+0000","last_active":"2024-04-18T23:11:51.889965+0000","last_peered":"2024-04-18T23:11:51.889965+0000","last_clean":"2024-04-18T23:11:51.889965+0000","last_became_active":"2024-04-18T23:10:54.712663+0000","last_became_peered":"2024-04-18T23:10:54.712663+0000","last_unstale":"2024-04-18T23:11:51.889965+0000","last_undegraded":"2024-04-18T23:11:51.889965+0000","last_fullsized":"2024-04-18T23:11:51.889965+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":126,"log_dups_size":0,"ondisk_log_size":126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":2,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:06.509474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.81427957100000004,"stat_sum":{"num_bytes":13123584,"num_objects":14,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":50,"num_read_kb":5296,"num_write":103,"num_write_kb":15088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.c","version":"81'51","reported_seq":301,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.764896+0000","last_change":"2024-04-18T23:11:50.865568+0000","last_active":"2024-04-18T23:11:50.764896+0000","last_peered":"2024-04-18T23:11:50.764896+0000","last_clean":"2024-04-18T23:11:50.764896+0000","last_became_active":"2024-04-18T23:10:54.717053+0000","last_became_peered":"2024-04-18T23:10:54.717053+0000","last_unstale":"2024-04-18T23:11:50.764896+0000","last_undegraded":"2024-04-18T23:11:50.764896+0000","last_fullsized":"2024-04-18T23:11:50.764896+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:54.955177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.101321669,"stat_sum":{"num_bytes":6815744,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":1926,"num_write":15,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.b","version":"84'133","reported_seq":381,"reported_epoch":84,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.473880+0000","last_change":"2024-04-18T23:11:50.644732+0000","last_active":"2024-04-18T23:11:51.473880+0000","last_peered":"2024-04-18T23:11:51.473880+0000","last_clean":"2024-04-18T23:11:51.473880+0000","last_became_active":"2024-04-18T23:10:54.808152+0000","last_became_peered":"2024-04-18T23:10:54.808152+0000","last_unstale":"2024-04-18T23:11:51.473880+0000","last_undegraded":"2024-04-18T23:11:51.473880+0000","last_fullsized":"2024-04-18T23:11:51.473880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:59.990562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.51132370100000002,"stat_sum":{"num_bytes":13631488,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":33,"num_read_kb":8320,"num_write":45,"num_write_kb":13584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"3.a","version":"84'20","reported_seq":280,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.227940+0000","last_change":"2024-04-18T23:11:50.820320+0000","last_active":"2024-04-18T23:11:51.227940+0000","last_peered":"2024-04-18T23:11:51.227940+0000","last_clean":"2024-04-18T23:11:51.227940+0000","last_became_active":"2024-04-18T23:10:54.764999+0000","last_became_peered":"2024-04-18T23:10:54.764999+0000","last_unstale":"2024-04-18T23:11:51.227940+0000","last_undegraded":"2024-04-18T23:11:51.227940+0000","last_fullsized":"2024-04-18T23:11:51.227940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:00.989345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10087621099999999,"stat_sum":{"num_bytes":720896,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":3750,"num_write":6,"num_write_kb":624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.9","version":"84'108","reported_seq":407,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.190183+0000","last_change":"2024-04-18T23:11:50.988961+0000","last_active":"2024-04-18T23:11:51.190183+0000","last_peered":"2024-04-18T23:11:51.190183+0000","last_clean":"2024-04-18T23:11:51.190183+0000","last_became_active":"2024-04-18T23:11:39.726824+0000","last_became_peered":"2024-04-18T23:11:39.726824+0000","last_unstale":"2024-04-18T23:11:51.190183+0000","last_undegraded":"2024-04-18T23:11:51.190183+0000","last_fullsized":"2024-04-18T23:11:51.190183+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":108,"log_dups_size":0,"ondisk_log_size":108,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:02.111340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.21817656799999999,"stat_sum":{"num_bytes":5816320,"num_objects":14,"num_object_clones":8,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":55,"num_read_kb":10769,"num_write":81,"num_write_kb":7696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2867200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.8","version":"84'55","reported_seq":331,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.869640+0000","last_change":"2024-04-18T23:11:50.871103+0000","last_active":"2024-04-18T23:11:50.869640+0000","last_peered":"2024-04-18T23:11:50.869640+0000","last_clean":"2024-04-18T23:11:50.869640+0000","last_became_active":"2024-04-18T23:11:37.876250+0000","last_became_peered":"2024-04-18T23:11:37.876250+0000","last_unstale":"2024-04-18T23:11:50.869640+0000","last_undegraded":"2024-04-18T23:11:50.869640+0000","last_fullsized":"2024-04-18T23:11:50.869640+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":55,"log_dups_size":0,"ondisk_log_size":55,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:17.115381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20782552300000001,"stat_sum":{"num_bytes":2129920,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":21,"num_read_kb":2250,"num_write":54,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"1.0","version":"33'157","reported_seq":336,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.647191+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:11:50.647191+0000","last_peered":"2024-04-18T23:11:50.647191+0000","last_clean":"2024-04-18T23:11:50.647191+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:11:50.647191+0000","last_undegraded":"2024-04-18T23:11:50.647191+0000","last_fullsized":"2024-04-18T23:11:50.647191+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":139,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.763430+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:11:50.763430+0000","last_peered":"2024-04-18T23:11:50.763430+0000","last_clean":"2024-04-18T23:11:50.763430+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:11:50.763430+0000","last_undegraded":"2024-04-18T23:11:50.763430+0000","last_fullsized":"2024-04-18T23:11:50.763430+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"84'65","reported_seq":338,"reported_epoch":84,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:50.982419+0000","last_change":"2024-04-18T23:11:50.881001+0000","last_active":"2024-04-18T23:11:50.982419+0000","last_peered":"2024-04-18T23:11:50.982419+0000","last_clean":"2024-04-18T23:11:50.982419+0000","last_became_active":"2024-04-18T23:11:33.818951+0000","last_became_peered":"2024-04-18T23:11:33.818951+0000","last_unstale":"2024-04-18T23:11:50.982419+0000","last_undegraded":"2024-04-18T23:11:50.982419+0000","last_fullsized":"2024-04-18T23:11:50.982419+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":65,"log_dups_size":0,"ondisk_log_size":65,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:22.058860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.523503465,"stat_sum":{"num_bytes":9797632,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":6012,"num_write":36,"num_write_kb":9872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":126,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.819088+0000","last_change":"2024-04-18T23:10:45.497972+0000","last_active":"2024-04-18T23:11:50.819088+0000","last_peered":"2024-04-18T23:11:50.819088+0000","last_clean":"2024-04-18T23:11:50.819088+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:11:50.819088+0000","last_undegraded":"2024-04-18T23:11:50.819088+0000","last_fullsized":"2024-04-18T23:11:50.819088+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:51.071639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"84'6","reported_seq":251,"reported_epoch":84,"state":"active+clean+snaptrim_wait","last_fresh":"2024-04-18T23:11:51.271218+0000","last_change":"2024-04-18T23:11:50.744771+0000","last_active":"2024-04-18T23:11:51.271218+0000","last_peered":"2024-04-18T23:11:51.271218+0000","last_clean":"2024-04-18T23:11:51.271218+0000","last_became_active":"2024-04-18T23:10:54.799189+0000","last_became_peered":"2024-04-18T23:10:54.799189+0000","last_unstale":"2024-04-18T23:11:51.271218+0000","last_undegraded":"2024-04-18T23:11:51.271218+0000","last_fullsized":"2024-04-18T23:11:51.271218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.458672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082261200000000002,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":126,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.647289+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:11:50.647289+0000","last_peered":"2024-04-18T23:11:50.647289+0000","last_clean":"2024-04-18T23:11:50.647289+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:11:50.647289+0000","last_undegraded":"2024-04-18T23:11:50.647289+0000","last_fullsized":"2024-04-18T23:11:50.647289+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:14.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"81'34","reported_seq":314,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.670683+0000","last_change":"2024-04-18T23:11:50.771452+0000","last_active":"2024-04-18T23:11:50.670683+0000","last_peered":"2024-04-18T23:11:50.670683+0000","last_clean":"2024-04-18T23:11:50.670683+0000","last_became_active":"2024-04-18T23:11:38.818022+0000","last_became_peered":"2024-04-18T23:11:38.818022+0000","last_unstale":"2024-04-18T23:11:50.670683+0000","last_undegraded":"2024-04-18T23:11:50.670683+0000","last_fullsized":"2024-04-18T23:11:50.670683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:17.808734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10106219900000001,"stat_sum":{"num_bytes":4407296,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":15,"num_read_kb":2791,"num_write":15,"num_write_kb":4304,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":7716864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":125,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.819341+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:11:50.819341+0000","last_peered":"2024-04-18T23:11:50.819341+0000","last_clean":"2024-04-18T23:11:50.819341+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:11:50.819341+0000","last_undegraded":"2024-04-18T23:11:50.819341+0000","last_fullsized":"2024-04-18T23:11:50.819341+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.994274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":260,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.647954+0000","last_change":"2024-04-18T23:11:50.648670+0000","last_active":"2024-04-18T23:11:50.647954+0000","last_peered":"2024-04-18T23:11:50.647954+0000","last_clean":"2024-04-18T23:11:50.647954+0000","last_became_active":"2024-04-18T23:11:33.713370+0000","last_became_peered":"2024-04-18T23:11:33.713370+0000","last_unstale":"2024-04-18T23:11:50.647954+0000","last_undegraded":"2024-04-18T23:11:50.647954+0000","last_fullsized":"2024-04-18T23:11:50.647954+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:04.176081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001062739,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.5","version":"84'98","reported_seq":375,"reported_epoch":85,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.679200+0000","last_change":"2024-04-18T23:11:51.679241+0000","last_active":"2024-04-18T23:11:51.679200+0000","last_peered":"2024-04-18T23:11:51.679200+0000","last_clean":"2024-04-18T23:11:51.679200+0000","last_became_active":"2024-04-18T23:11:39.923911+0000","last_became_peered":"2024-04-18T23:11:39.923911+0000","last_unstale":"2024-04-18T23:11:51.679200+0000","last_undegraded":"2024-04-18T23:11:51.679200+0000","last_fullsized":"2024-04-18T23:11:51.679200+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:56.743945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":1.0097257239999999,"stat_sum":{"num_bytes":7716864,"num_objects":11,"num_object_clones":8,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":39,"num_read_kb":12301,"num_write":51,"num_write_kb":9744,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5685248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":134,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.644455+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:11:50.644455+0000","last_peered":"2024-04-18T23:11:50.644455+0000","last_clean":"2024-04-18T23:11:50.644455+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:11:50.644455+0000","last_undegraded":"2024-04-18T23:11:50.644455+0000","last_fullsized":"2024-04-18T23:11:50.644455+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"84'82","reported_seq":322,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.982890+0000","last_change":"2024-04-18T23:11:50.983789+0000","last_active":"2024-04-18T23:11:50.982890+0000","last_peered":"2024-04-18T23:11:50.982890+0000","last_clean":"2024-04-18T23:11:50.982890+0000","last_became_active":"2024-04-18T23:10:54.793319+0000","last_became_peered":"2024-04-18T23:10:54.793319+0000","last_unstale":"2024-04-18T23:11:50.982890+0000","last_undegraded":"2024-04-18T23:11:50.982890+0000","last_fullsized":"2024-04-18T23:11:50.982890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":82,"log_dups_size":0,"ondisk_log_size":82,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:11.752239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30504247200000001,"stat_sum":{"num_bytes":3063808,"num_objects":9,"num_object_clones":5,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":3,"num_read":14,"num_read_kb":6,"num_write":23,"num_write_kb":3408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":126,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.661941+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:11:50.661941+0000","last_peered":"2024-04-18T23:11:50.661941+0000","last_clean":"2024-04-18T23:11:50.661941+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:11:50.661941+0000","last_undegraded":"2024-04-18T23:11:50.661941+0000","last_fullsized":"2024-04-18T23:11:50.661941+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.503236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"84'182","reported_seq":425,"reported_epoch":84,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:11:51.476470+0000","last_change":"2024-04-18T23:11:50.657733+0000","last_active":"2024-04-18T23:11:51.476470+0000","last_peered":"2024-04-18T23:11:51.476470+0000","last_clean":"2024-04-18T23:11:51.476470+0000","last_became_active":"2024-04-18T23:10:54.693609+0000","last_became_peered":"2024-04-18T23:10:54.693609+0000","last_unstale":"2024-04-18T23:11:51.476470+0000","last_undegraded":"2024-04-18T23:11:51.476470+0000","last_fullsized":"2024-04-18T23:11:51.476470+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":182,"log_dups_size":0,"ondisk_log_size":182,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:13.905711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.472976709,"stat_sum":{"num_bytes":13500416,"num_objects":17,"num_object_clones":9,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":47,"num_read_kb":11702,"num_write":163,"num_write_kb":14880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"18","length":"1"},{"start":"1a","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":125,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.678768+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:11:50.678768+0000","last_peered":"2024-04-18T23:11:50.678768+0000","last_clean":"2024-04-18T23:11:50.678768+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:11:50.678768+0000","last_undegraded":"2024-04-18T23:11:50.678768+0000","last_fullsized":"2024-04-18T23:11:50.678768+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.869947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"83'40","reported_seq":321,"reported_epoch":84,"state":"active+clean","last_fresh":"2024-04-18T23:11:50.660532+0000","last_change":"2024-04-18T23:11:50.761229+0000","last_active":"2024-04-18T23:11:50.660532+0000","last_peered":"2024-04-18T23:11:50.660532+0000","last_clean":"2024-04-18T23:11:50.660532+0000","last_became_active":"2024-04-18T23:11:39.717443+0000","last_became_peered":"2024-04-18T23:11:39.717443+0000","last_unstale":"2024-04-18T23:11:50.660532+0000","last_undegraded":"2024-04-18T23:11:50.660532+0000","last_fullsized":"2024-04-18T23:11:50.660532+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:00.941867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.100906204,"stat_sum":{"num_bytes":3866624,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":1879,"num_write":18,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":54,"reported_epoch":85,"state":"active+clean","last_fresh":"2024-04-18T23:11:51.662204+0000","last_change":"2024-04-18T23:11:33.519671+0000","last_active":"2024-04-18T23:11:51.662204+0000","last_peered":"2024-04-18T23:11:51.662204+0000","last_clean":"2024-04-18T23:11:51.662204+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:11:51.662204+0000","last_undegraded":"2024-04-18T23:11:51.662204+0000","last_fullsized":"2024-04-18T23:11:51.662204+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":93552640,"num_objects":114,"num_object_clones":64,"num_object_copies":798,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":6,"num_read":394,"num_read_kb":96725,"num_write":674,"num_write_kb":104352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":163749888,"data_stored":211496960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0},"log_size":1095,"ondisk_log_size":1095,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808837,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38580,"kb_used_data":10016,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94333260,"statfs":{"total":96636764160,"available":96597258240,"internally_reserved":0,"allocated":10256384,"data_stored":11702167,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,1,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,2,0,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280530,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39968,"kb_used_data":8784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331872,"statfs":{"total":96636764160,"available":96595836928,"internally_reserved":0,"allocated":8994816,"data_stored":9893638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280530,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39336,"kb_used_data":8156,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332504,"statfs":{"total":96636764160,"available":96596484096,"internally_reserved":0,"allocated":8351744,"data_stored":9636134,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313233,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49716,"kb_used_data":18536,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322124,"statfs":{"total":96636764160,"available":96585854976,"internally_reserved":0,"allocated":18980864,"data_stored":22669062,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313233,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44660,"kb_used_data":13480,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327180,"statfs":{"total":96636764160,"available":96591032320,"internally_reserved":0,"allocated":13803520,"data_stored":16643846,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313234,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34872,"kb_used_data":7856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336968,"statfs":{"total":96636764160,"available":96601055232,"internally_reserved":0,"allocated":8044544,"data_stored":9104979,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313234,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45404,"kb_used_data":14224,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326436,"statfs":{"total":96636764160,"available":96590270464,"internally_reserved":0,"allocated":14565376,"data_stored":17917702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,7],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313234,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33976,"kb_used_data":6960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94337864,"statfs":{"total":96636764160,"available":96601972736,"internally_reserved":0,"allocated":7127040,"data_stored":7376467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,7,0,0,1],"upper_bound":2048},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313232,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45784,"kb_used_data":14604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326056,"statfs":{"total":96636764160,"available":96589881344,"internally_reserved":0,"allocated":14954496,"data_stored":18700038,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,3,0,3],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313232,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37392,"kb_used_data":10376,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334448,"statfs":{"total":96636764160,"available":96598474752,"internally_reserved":0,"allocated":10625024,"data_stored":11779667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313233,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39996,"kb_used_data":8816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331844,"statfs":{"total":96636764160,"available":96595808256,"internally_reserved":0,"allocated":9027584,"data_stored":10489427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,1],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280530,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39792,"kb_used_data":8616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332048,"statfs":{"total":96636764160,"available":96596017152,"internally_reserved":0,"allocated":8822784,"data_stored":9977971,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280530,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47596,"kb_used_data":16420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324244,"statfs":{"total":96636764160,"available":96588025856,"internally_reserved":0,"allocated":16814080,"data_stored":20958803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,4,1,1],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280531,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41904,"kb_used_data":10720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329936,"statfs":{"total":96636764160,"available":96593854464,"internally_reserved":0,"allocated":10977280,"data_stored":13291415,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280530,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41792,"kb_used_data":10608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330048,"statfs":{"total":96636764160,"available":96593969152,"internally_reserved":0,"allocated":10862592,"data_stored":14059270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280530,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39160,"kb_used_data":7972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332680,"statfs":{"total":96636764160,"available":96596664320,"internally_reserved":0,"allocated":8163328,"data_stored":8720279,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":17879040,"data_stored":22052864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12587008,"data_stored":17158144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":7090176,"data_stored":9932800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7344128,"data_stored":8962048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":13869056,"data_stored":17727488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6701056,"data_stored":7847936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16388096,"data_stored":22294528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9433088,"data_stored":12075008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":14721024,"data_stored":19382272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9666560,"data_stored":13377536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6930432,"data_stored":8007680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":9908224,"data_stored":13148160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":7827456,"data_stored":10964992,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":6602752,"data_stored":8413184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":7778304,"data_stored":9162752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":9023488,"data_stored":10989568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0}]}} 2024-04-18T23:11:53.639 INFO:tasks.ceph.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-04-18T23:11:53.673 INFO:tasks.rados.rados.0.smithi142.stdout:318: done (3 left) 2024-04-18T23:11:53.673 INFO:tasks.rados.rados.0.smithi142.stdout:319: setattr oid 20 current snap is 27 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:316: finishing write tid 1 to smithi14230322-18 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:316: finishing write tid 2 to smithi14230322-18 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:316: finishing write tid 3 to smithi14230322-18 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 45 v 145 (ObjNum 102 snap 25 seq_num 102) dirty exists 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: finishing write tid 1 to smithi14230322-16 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: finishing write tid 2 to smithi14230322-16 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: finishing write tid 3 to smithi14230322-16 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: finishing write tid 4 to smithi14230322-16 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: finishing write tid 5 to smithi14230322-16 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: finishing write tid 6 to smithi14230322-16 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: oid 16 updating version 0 to 40 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: oid 16 updating version 40 to 42 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: oid 16 version 42 is already newer than 41 2024-04-18T23:11:53.674 INFO:tasks.rados.rados.0.smithi142.stdout:315: oid 16 version 42 is already newer than 39 2024-04-18T23:11:53.675 INFO:tasks.rados.rados.0.smithi142.stdout:315: oid 16 version 42 is already newer than 38 2024-04-18T23:11:53.675 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 16 v 42 (ObjNum 120 snap 27 seq_num 120) dirty exists 2024-04-18T23:11:53.675 INFO:tasks.rados.rados.0.smithi142.stdout:315: left oid 16 (ObjNum 120 snap 27 seq_num 120) 2024-04-18T23:11:53.675 INFO:tasks.rados.rados.0.smithi142.stdout:316: finishing write tid 4 to smithi14230322-18 2024-04-18T23:11:53.675 INFO:tasks.rados.rados.0.smithi142.stdout:316: finishing write tid 5 to smithi14230322-18 2024-04-18T23:11:53.676 INFO:tasks.rados.rados.0.smithi142.stdout:315: done (3 left) 2024-04-18T23:11:53.677 INFO:tasks.rados.rados.0.smithi142.stdout:317: done (2 left) 2024-04-18T23:11:53.677 INFO:tasks.rados.rados.0.smithi142.stdout:320: snap_create 2024-04-18T23:11:53.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T23:11:53.719 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:53.720 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:53.720 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:53.718+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:53.729 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:53.729 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:53.734 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:11:53.766 INFO:tasks.rados.rados.0.smithi142.stdout:316: finishing write tid 6 to smithi14230322-18 2024-04-18T23:11:53.766 INFO:tasks.rados.rados.0.smithi142.stdout:316: oid 18 updating version 0 to 91 2024-04-18T23:11:53.766 INFO:tasks.rados.rados.0.smithi142.stdout:316: oid 18 updating version 91 to 92 2024-04-18T23:11:53.766 INFO:tasks.rados.rados.0.smithi142.stdout:316: oid 18 updating version 92 to 93 2024-04-18T23:11:53.766 INFO:tasks.rados.rados.0.smithi142.stdout:316: oid 18 updating version 93 to 94 2024-04-18T23:11:53.767 INFO:tasks.rados.rados.0.smithi142.stdout:316: oid 18 updating version 94 to 95 2024-04-18T23:11:53.767 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 18 v 95 (ObjNum 121 snap 27 seq_num 121) dirty exists 2024-04-18T23:11:53.767 INFO:tasks.rados.rados.0.smithi142.stdout:316: left oid 18 (ObjNum 121 snap 27 seq_num 121) 2024-04-18T23:11:53.811 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:53.810+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:53.878 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:53.878 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:53.894 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:53.894 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:53.900 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:53.900 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:53.908 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:11:53.913 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:53.913+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:53.922 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:53.923 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:53.998 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:53.998 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:54.014 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:54.011+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:54.017 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:54.017 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:54.018 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:54.018 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:54.031 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:11:54.037 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:54.038 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:54.058 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:54.058 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:54.085 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:54.085 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:54.115 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:54.113+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:54.146 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:54.146 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:54.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:11:54.185 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 20 v 57 (ObjNum 95 snap 21 seq_num 95) dirty exists 2024-04-18T23:11:54.189 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:54.190 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:54.214 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:54.215 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:54.215 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:54.210+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:54.238 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:54.239 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:54.259 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:54.260 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:54.300 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:54.300 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:11:54.300 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:11:54.300 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:11:54.301 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:54.313 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:11:54.316 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:54.315+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:54.316 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:54.316 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:54.337 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:54.338 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:54.359 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:54.360 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:54.416 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:54.415+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:54.430 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:11:54.430 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:11:54.430 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:11:54.430 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:11:54.430 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.430 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 34/31 ECSubWrite(tid=1, reqid=client.4568.0:38, at_version=34'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.805396923000004, 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10419479800000001, 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.431 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624028+0000", 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624029+0000", 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4300000000000001e-07 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.432 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624060+0000", 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1025999999999997e-05 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624083+0000", 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3323e-05 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.624104+0000", 2024-04-18T23:11:54.433 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1089000000000001e-05 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725186+0000", 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101081146 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.725195+0000", 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7829999999999993e-06 2024-04-18T23:11:54.434 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728164+0000", 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0029689249999999999 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:10:56.728223+0000", 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8563000000000002e-05 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.435 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 42/31 ECSubWrite(tid=85, reqid=osd.4.0:85, at_version=42'27, trim_to=0'0, roll_forward_to=40'25))", 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 47.851692651, 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16837443799999999, 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.436 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577733+0000", 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6300000000000001e-07 2024-04-18T23:11:54.437 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577758+0000", 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5099000000000001e-05 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577779+0000", 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0937999999999998e-05 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.438 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577792+0000", 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3227999999999999e-05 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744736+0000", 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.166943963 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744769+0000", 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2629000000000002e-05 2024-04-18T23:11:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746075+0000", 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001306338 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746107+0000", 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.188e-05 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.440 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 47/31 ECSubWrite(tid=26, reqid=client.4568.0:333, at_version=47'16, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.697540889000003, 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201860068, 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.442 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6299999999999999e-07 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732265+0000", 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038060399999999999 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732281+0000", 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6362e-05 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732292+0000", 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0817e-05 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.832471+0000", 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100179254 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.832486+0000", 2024-04-18T23:11:54.444 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4911e-05 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933708+0000", 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101221567 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933744+0000", 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6390000000000002e-05 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.445 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 47/31 ECSubWrite(tid=27, reqid=client.4568.0:344, at_version=47'18, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 41.694893555, 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19924303900000001, 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.447 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734532+0000", 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734555+0000", 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3258999999999999e-05 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734568+0000", 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2568999999999999e-05 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734575+0000", 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6279999999999999e-06 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.833597+0000", 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099021993000000003 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.449 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.833605+0000", 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0620000000000001e-06 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933755+0000", 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10015038 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933774+0000", 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9474999999999999e-05 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.450 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 60/31 ECSubWrite(tid=38, reqid=client.4568.0:452, at_version=0'0, trim_to=0'0, roll_forward_to=60'10))", 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.352985+0000", 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 28.076440409, 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30133912600000001, 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.352985+0000", 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.352985+0000", 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.452 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.352986+0000", 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0109999999999999e-06 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.353002+0000", 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6235999999999998e-05 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.353020+0000", 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8451999999999999e-05 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.453 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.453102+0000", 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100081295 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.553350+0000", 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100248296 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.553384+0000", 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3729000000000002e-05 2024-04-18T23:11:54.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654230+0000", 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100846175 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654324+0000", 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3931999999999994e-05 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 61/31 ECSubWrite(tid=183, reqid=osd.4.0:183, at_version=61'66, trim_to=0'0, roll_forward_to=58'62))", 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.746721+0000", 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.682703776, 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104157869, 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.456 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746721+0000", 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746721+0000", 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746723+0000", 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.302e-06 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.457 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746748+0000", 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5795999999999998e-05 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.746766+0000", 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7342000000000001e-05 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.458 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.846876+0000", 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10010988 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847103+0000", 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000227848 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847132+0000", 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8132000000000001e-05 2024-04-18T23:11:54.459 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.850847+0000", 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0037155199999999999 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.850879+0000", 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2048999999999998e-05 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 61/31 ECSubWrite(tid=43, reqid=osd.2.0:43, at_version=61'31, trim_to=0'0, roll_forward_to=60'27))", 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.847226+0000", 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.582198511000001, 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201633386, 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.461 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847226+0000", 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847226+0000", 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847227+0000", 2024-04-18T23:11:54.462 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847269+0000", 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2058999999999998e-05 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847292+0000", 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2299000000000001e-05 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.947376+0000", 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008486599999999 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.047557+0000", 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10018052099999999 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.047574+0000", 2024-04-18T23:11:54.464 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6690000000000001e-05 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.048752+0000", 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011779469999999999 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.048860+0000", 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108194 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.465 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 72/31 ECSubWrite(tid=65, reqid=osd.2.0:65, at_version=72'50, trim_to=0'0, roll_forward_to=71'48))", 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.617025+0000", 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.812399502, 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20153253099999999, 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.466 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617025+0000", 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617025+0000", 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.467 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617026+0000", 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8999999999999996e-07 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617083+0000", 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7328999999999999e-05 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617107+0000", 2024-04-18T23:11:54.468 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4099000000000001e-05 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617124+0000", 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6310000000000001e-05 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.817365+0000", 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20024151300000001 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.469 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.817375+0000", 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8660000000000001e-06 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.818514+0000", 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011390809999999999 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.818558+0000", 2024-04-18T23:11:54.470 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3742999999999998e-05 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 72/31 ECSubWrite(tid=66, reqid=osd.2.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.618834+0000", 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.810590787000001, 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20101856300000001, 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.471 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618834+0000", 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618834+0000", 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618835+0000", 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618863+0000", 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7520000000000001e-05 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.473 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618882+0000", 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9066e-05 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.718989+0000", 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100107504 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.817858+0000", 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.098869033999999995 2024-04-18T23:11:54.474 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.817869+0000", 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0997e-05 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819809+0000", 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019393909999999999 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.475 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819853+0000", 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4155e-05 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 72/31 ECSubWrite(tid=60, reqid=client.4568.0:561, at_version=72'28, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.630952+0000", 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.798473263, 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.174762375, 2024-04-18T23:11:54.476 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.630952+0000", 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.630952+0000", 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.477 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.630952+0000", 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6700000000000003e-07 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.631436+0000", 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00048351399999999999 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.478 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.631467+0000", 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1149999999999998e-05 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.631483+0000", 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6348999999999999e-05 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.704225+0000", 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.072741410000000006 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.479 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.704236+0000", 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0828e-05 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805658+0000", 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101421936 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805714+0000", 2024-04-18T23:11:54.480 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6421000000000001e-05 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 72/31 ECSubWrite(tid=61, reqid=osd.11.0:61, at_version=72'30, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.634135+0000", 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.795290074, 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.17163550899999999, 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.481 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634135+0000", 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634135+0000", 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634136+0000", 2024-04-18T23:11:54.482 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3829999999999999e-06 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634172+0000", 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5886000000000002e-05 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634197+0000", 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5018999999999999e-05 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.483 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634214+0000", 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6439999999999998e-05 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.704741+0000", 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.070527272000000002 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.704748+0000", 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5719999999999997e-06 2024-04-18T23:11:54.484 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805721+0000", 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100973006 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805770+0000", 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8931000000000003e-05 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.485 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 72/31 ECSubWrite(tid=63, reqid=osd.11.0:63, at_version=72'32, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.636571+0000", 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.792853609, 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.169977613, 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.486 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636571+0000", 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636571+0000", 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636573+0000", 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5689999999999999e-06 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.487 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636610+0000", 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6946000000000002e-05 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636646+0000", 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6298999999999999e-05 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.488 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636667+0000", 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0429999999999999e-05 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805168+0000", 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16850090300000001 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.489 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805182+0000", 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4132e-05 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.806507+0000", 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001325718 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.806549+0000", 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1616000000000003e-05 2024-04-18T23:11:54.490 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 72/31 ECSubWrite(tid=64, reqid=client.4568.0:562, at_version=72'33, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.639938+0000", 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.789486762999999, 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16704977500000001, 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.491 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.639938+0000", 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.639938+0000", 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.639939+0000", 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2529999999999999e-06 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.640192+0000", 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000252348 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.493 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.640216+0000", 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3808999999999999e-05 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.640230+0000", 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4114000000000001e-05 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805831+0000", 2024-04-18T23:11:54.494 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16560166500000001 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805844+0000", 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3044000000000001e-05 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.806968+0000", 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001123815 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.495 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.806988+0000", 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9726999999999999e-05 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 72/31 ECSubWrite(tid=215, reqid=osd.4.0:215, at_version=72'74, trim_to=0'0, roll_forward_to=71'72))", 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.719128+0000", 2024-04-18T23:11:54.496 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.710297028999999, 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20248240000000001, 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719128+0000", 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.497 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719128+0000", 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719129+0000", 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6700000000000003e-07 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719183+0000", 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4279e-05 2024-04-18T23:11:54.498 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.499 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.499 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.499 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719207+0000", 2024-04-18T23:11:54.499 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4586000000000001e-05 2024-04-18T23:11:54.499 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.501 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.501 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.501 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819295+0000", 2024-04-18T23:11:54.501 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008806200000001 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.919464+0000", 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100168431 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.919480+0000", 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5722999999999998e-05 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.502 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.921452+0000", 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001972054 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.921610+0000", 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000158698 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.503 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 72/31 ECSubWrite(tid=217, reqid=osd.4.0:217, at_version=72'76, trim_to=0'0, roll_forward_to=71'72))", 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.819325+0000", 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.610100357, 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10487545299999999, 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819325+0000", 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.504 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819325+0000", 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819325+0000", 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5199999999999997e-07 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.505 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819356+0000", 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0358e-05 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819369+0000", 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3896999999999999e-05 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819379+0000", 2024-04-18T23:11:54.506 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0047000000000001e-05 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.919961+0000", 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100581476 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.919973+0000", 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1758e-05 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.507 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.924166+0000", 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041929030000000004 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.924200+0000", 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4462000000000003e-05 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.508 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 78/31 ECSubWrite(tid=74, reqid=osd.11.0:74, at_version=78'41, trim_to=0'0, roll_forward_to=73'39))", 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:44.613875+0000", 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.8155502840000004, 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104358825, 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.613875+0000", 2024-04-18T23:11:54.509 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.613875+0000", 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.613875+0000", 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8700000000000005e-07 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.510 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.613933+0000", 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7669e-05 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.613970+0000", 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7042000000000002e-05 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.511 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.614001+0000", 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1306000000000001e-05 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.714301+0000", 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100299551 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.714317+0000", 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5913000000000002e-05 2024-04-18T23:11:54.512 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.717132+0000", 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0028147950000000001 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.718234+0000", 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011018619999999999 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.513 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 78/31 ECSubWrite(tid=263, reqid=client.4568.0:639, at_version=78'93, trim_to=0'0, roll_forward_to=77'87))", 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:44.759747+0000", 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.6696776979999992, 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10655000000000001, 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.514 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.759747+0000", 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.759747+0000", 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.759748+0000", 2024-04-18T23:11:54.515 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7300000000000005e-07 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.759916+0000", 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016770199999999999 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.759957+0000", 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1143000000000002e-05 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.516 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.759980+0000", 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2671999999999999e-05 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.860146+0000", 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100166404 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.860152+0000", 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2469999999999997e-06 2024-04-18T23:11:54.517 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.866249+0000", 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0060971319999999999 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.866297+0000", 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7926999999999997e-05 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.518 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 81/31 ECSubWrite(tid=309, reqid=osd.4.0:305, at_version=0'0, trim_to=0'0, roll_forward_to=81'120))", 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:48.545814+0000", 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.8836109250000002, 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20083568399999999, 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.519 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545814+0000", 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545814+0000", 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.520 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545815+0000", 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3969999999999999e-06 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545853+0000", 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8000999999999998e-05 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545874+0000", 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0068000000000002e-05 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.521 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545891+0000", 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7176999999999998e-05 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.646066+0000", 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10017543700000001 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.522 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.646074+0000", 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9389999999999997e-06 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.746609+0000", 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100534928 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.746650+0000", 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0737e-05 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.523 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 83/31 ECSubWrite(tid=318, reqid=client.4568.0:695, at_version=83'126, trim_to=0'0, roll_forward_to=82'124))", 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.829904+0000", 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 4.5995206670000002, 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.104155815, 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.524 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.829904+0000", 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.829904+0000", 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.829905+0000", 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3000000000000001e-07 2024-04-18T23:11:54.525 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.830200+0000", 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029467700000000002 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.830223+0000", 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3558999999999999e-05 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.526 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.830239+0000", 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6291999999999999e-05 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.830400+0000", 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016038000000000001 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.830429+0000", 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9034999999999999e-05 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.527 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.934007+0000", 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103578143 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.934060+0000", 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3198999999999997e-05 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.528 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.bs3 84/31 ECSubRead(tid=332, to_read={3:d447fef0:::smithi14230322-49:head=0,4096,0}, subchunks={3:d447fef0:::smithi14230322-49:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.759540+0000", 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.6698849930000002, 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.200539947, 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759540+0000", 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.529 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759540+0000", 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759541+0000", 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9999999999999997e-07 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.530 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759548+0000", 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5000000000000002e-06 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759559+0000", 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0587999999999999e-05 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.859637+0000", 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100078449 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.531 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.959780+0000", 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100142606 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.960080+0000", 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030010399999999998 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:11:54.532 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:11:54.533 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:11:54.534 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:11:54.535 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:54.530+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:54.635 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:54.635+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:54.664 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 87/31 ECSubWrite(tid=383, reqid=osd.4.0:379, at_version=0'0, trim_to=0'0, roll_forward_to=87'153))", 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:54.574090+0000", 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.092415038000000005, 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.092435258000000006, 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "no flag points reached", 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.665 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.574090+0000", 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.574090+0000", 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.666 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.574093+0000", 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2199999999999999e-06 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.574121+0000", 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7915999999999998e-05 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.574154+0000", 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2947999999999997e-05 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.667 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.668 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.668 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.668 INFO:teuthology.orchestra.run.smithi120.stdout: ], 2024-04-18T23:11:54.668 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 1 2024-04-18T23:11:54.668 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:54.671 INFO:tasks.rados.rados.0.smithi142.stdout:316: done (2 left) 2024-04-18T23:11:54.672 INFO:tasks.rados.rados.0.smithi142.stdout:319: done (1 left) 2024-04-18T23:11:54.672 INFO:tasks.rados.rados.0.smithi142.stdout:320: done (0 left) 2024-04-18T23:11:54.672 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 33 current snap is 28 2024-04-18T23:11:54.672 INFO:tasks.rados.rados.0.smithi142.stdout:321: seq_num 122 ranges {1228800=507904,1736704=425984,2162688=442368,2605056=540672,3145728=147456} 2024-04-18T23:11:54.672 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-04-18T23:11:54.681 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:11:54.688 INFO:tasks.rados.rados.0.smithi142.stdout:321: writing smithi14230322-33 from 1228800 to 1736704 tid 1 2024-04-18T23:11:54.692 INFO:tasks.rados.rados.0.smithi142.stdout:321: writing smithi14230322-33 from 1736704 to 2162688 tid 2 2024-04-18T23:11:54.696 INFO:tasks.rados.rados.0.smithi142.stdout:321: writing smithi14230322-33 from 2162688 to 2605056 tid 3 2024-04-18T23:11:54.701 INFO:tasks.rados.rados.0.smithi142.stdout:321: writing smithi14230322-33 from 2605056 to 3145728 tid 4 2024-04-18T23:11:54.703 INFO:tasks.rados.rados.0.smithi142.stdout:321: writing smithi14230322-33 from 3145728 to 3293184 tid 5 2024-04-18T23:11:54.703 INFO:tasks.rados.rados.0.smithi142.stdout:322: copy_from oid 18 from oid 13 current snap is 28 2024-04-18T23:11:54.703 INFO:tasks.rados.rados.0.smithi142.stdout:323: snap_create 2024-04-18T23:11:54.736 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:54.735+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:54.808 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:54.808 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:54.808 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:54.808 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:54.808 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:54.822 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:11:54.836 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:54.835+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:54.902 INFO:tasks.rados.rados.0.smithi142.stdout:321: finishing write tid 1 to smithi14230322-33 2024-04-18T23:11:54.920 INFO:tasks.rados.rados.0.smithi142.stdout:322: finishing copy_from racing read to smithi14230322-18 2024-04-18T23:11:54.937 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:54.935+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:54.946 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 61/31 ECSubWrite(tid=186, reqid=osd.4.0:186, at_version=61'68, trim_to=0'0, roll_forward_to=61'66))", 2024-04-18T23:11:54.946 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.990144564000001, 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201499973, 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956661+0000", 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7370000000000001e-06 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956723+0000", 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2564999999999995e-05 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956762+0000", 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8711999999999998e-05 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.056877+0000", 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10011492900000001 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.057020+0000", 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142662 2024-04-18T23:11:54.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.057046+0000", 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6486e-05 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.158121+0000", 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10107499 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.158159+0000", 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7892000000000002e-05 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 61/31 ECSubWrite(tid=45, reqid=osd.2.0:45, at_version=61'33, trim_to=0'0, roll_forward_to=61'31))", 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:11:54.951 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.601216751999999, 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20243294100000001, 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345589+0000", 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7540000000000001e-06 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345643+0000", 2024-04-18T23:11:54.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4707999999999999e-05 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345662+0000", 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8870000000000001e-05 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345675+0000", 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2482e-05 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.545813+0000", 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20013862499999999 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.545829+0000", 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5201999999999999e-05 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.547933+0000", 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021040389999999998 2024-04-18T23:11:54.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.548020+0000", 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7261000000000004e-05 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 70/31 ECSubWrite(tid=57, reqid=client.4568.0:551, at_version=70'43, trim_to=0'0, roll_forward_to=69'42))", 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.565666+0000", 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.381137867, 2024-04-18T23:11:54.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202564679, 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565666+0000", 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.957 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565666+0000", 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565667+0000", 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2699999999999998e-07 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565871+0000", 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020396499999999999 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565899+0000", 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8572e-05 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.666000+0000", 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100101172 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.766291+0000", 2024-04-18T23:11:54.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100290465 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.766307+0000", 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5979999999999999e-05 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.768156+0000", 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018494799999999999 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.768230+0000", 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4218e-05 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 70/31 ECSubWrite(tid=61, reqid=client.4568.0:554, at_version=0'0, trim_to=0'0, roll_forward_to=70'46))", 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.082659+0000", 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.864144640999999, 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20138941299999999, 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.961 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082659+0000", 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082659+0000", 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082661+0000", 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.694e-06 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082682+0000", 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1231999999999999e-05 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082712+0000", 2024-04-18T23:11:54.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0273e-05 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.182872+0000", 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100159647 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.183044+0000", 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000172356 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.183071+0000", 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7097000000000002e-05 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.283975+0000", 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100903359 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.284048+0000", 2024-04-18T23:11:54.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3755000000000001e-05 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.9s1 71/68 ECSubRead(tid=147, to_read={3:9e84ac80:::smithi14230322-40:16=0,262144,0}, subchunks={3:9e84ac80:::smithi14230322-40:16=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.680353+0000", 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.266450659, 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20058264000000001, 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.966 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680353+0000", 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680353+0000", 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680355+0000", 2024-04-18T23:11:54.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.716e-06 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680372+0000", 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7346999999999998e-05 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.680405+0000", 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2685999999999998e-05 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.780512+0000", 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100106973 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.880742+0000", 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100230139 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.880936+0000", 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019377899999999999 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.969 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.8s3 72/70 ECSubWrite(tid=40, reqid=osd.5.0:39, at_version=0'0, trim_to=0'0, roll_forward_to=72'30))", 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:39.022358+0000", 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 15.924445302000001, 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20051693100000001, 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.022358+0000", 2024-04-18T23:11:54.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.022358+0000", 2024-04-18T23:11:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.022359+0000", 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.64e-07 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.022375+0000", 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5174000000000001e-05 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.022391+0000", 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6541e-05 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.122469+0000", 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100077755 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.122687+0000", 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00021842399999999999 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.122705+0000", 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7863000000000002e-05 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.222824+0000", 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10011840800000001 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.974 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.222875+0000", 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.1802000000000003e-05 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 77/31 ECSubWrite(tid=247, reqid=client.4568.0:607, at_version=77'85, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.821380+0000", 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.125423261, 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20502453600000001, 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.975 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821380+0000", 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821380+0000", 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.976 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821382+0000", 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.973e-06 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821601+0000", 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000219053 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821630+0000", 2024-04-18T23:11:54.977 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8651999999999999e-05 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921792+0000", 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10016209 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921968+0000", 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017578900000000001 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.978 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921988+0000", 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0029000000000001e-05 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.026322+0000", 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10433400499999999 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.979 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.026405+0000", 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2945e-05 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 77/31 ECSubWrite(tid=248, reqid=client.4568.0:608, at_version=77'86, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.921849+0000", 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.024954927, 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20508053000000001, 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.980 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921849+0000", 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921849+0000", 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.981 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921850+0000", 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0839999999999999e-06 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921982+0000", 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132464 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922007+0000", 2024-04-18T23:11:54.982 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5089e-05 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922021+0000", 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3113e-05 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.923052+0000", 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010313989999999999 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.983 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.923081+0000", 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9125e-05 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126891+0000", 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.203809765 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126929+0000", 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8491000000000001e-05 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.984 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 77/31 ECSubWrite(tid=249, reqid=client.4568.0:609, at_version=77'87, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.922042+0000", 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.024761889000001, 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20493312899999999, 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.985 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922042+0000", 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922042+0000", 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922042+0000", 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-04-18T23:11:54.986 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922064+0000", 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1611e-05 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922076+0000", 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1732999999999999e-05 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.987 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922085+0000", 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0259999999999996e-06 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.924159+0000", 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020747249999999999 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.924190+0000", 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0151e-05 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.988 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126939+0000", 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20274966799999999 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126975+0000", 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5680999999999999e-05 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.989 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.bs2 77/31 ECSubRead(tid=254, to_read={3:d447fef0:::smithi14230322-49:16=0,475136,0}, subchunks={3:d447fef0:::smithi14230322-49:16=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.922199+0000", 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.024604845000001, 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.204785145, 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922199+0000", 2024-04-18T23:11:54.990 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922199+0000", 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922199+0000", 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2000000000000001e-07 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.991 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922204+0000", 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2290000000000002e-06 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922219+0000", 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4676e-05 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922234+0000", 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4763000000000001e-05 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.992 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126655+0000", 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20442090399999999 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126984+0000", 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000329253 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.993 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 81/72 ECSubWrite(tid=187, reqid=osd.14.0:187, at_version=81'90, trim_to=0'0, roll_forward_to=80'88))", 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:47.647535+0000", 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 7.2992688130000003, 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201612076, 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647535+0000", 2024-04-18T23:11:54.994 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647535+0000", 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647536+0000", 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2399999999999997e-07 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:54.995 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647553+0000", 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7581999999999999e-05 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647565+0000", 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1487e-05 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.747635+0000", 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100070545 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:54.996 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.747765+0000", 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013010499999999999 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.747782+0000", 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6935999999999998e-05 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.849112+0000", 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10133017900000001 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.997 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.849147+0000", 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4517999999999997e-05 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 81/31 ECSubWrite(tid=299, reqid=osd.4.0:299, at_version=81'116, trim_to=0'0, roll_forward_to=80'112))", 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:47.737322+0000", 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 7.2094819279999998, 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201559339, 2024-04-18T23:11:54.998 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737322+0000", 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737322+0000", 2024-04-18T23:11:54.999 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737322+0000", 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9999999999999997e-07 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737355+0000", 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2598000000000002e-05 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.000 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737368+0000", 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2702e-05 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.837446+0000", 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100078242 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.937649+0000", 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100203006 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.001 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.937683+0000", 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4464000000000001e-05 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.938838+0000", 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001154293 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.002 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.938881+0000", 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3433999999999998e-05 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.6s2 81/72 ECSubWrite(tid=49, reqid=osd.0.0:48, at_version=0'0, trim_to=0'0, roll_forward_to=81'36))", 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.042478+0000", 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.904325633, 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20112732899999999, 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.003 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042478+0000", 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042478+0000", 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.004 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042479+0000", 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1969999999999999e-06 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042496+0000", 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6589e-05 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042548+0000", 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2468e-05 2024-04-18T23:11:55.005 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.142626+0000", 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100077679 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.142727+0000", 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010097799999999999 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.006 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.142744+0000", 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6801999999999999e-05 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.243552+0000", 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10080777 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.243605+0000", 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3845999999999999e-05 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.007 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.as2 81/31 ECSubWrite(tid=33, reqid=osd.3.0:32, at_version=0'0, trim_to=0'0, roll_forward_to=81'18))", 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.046379+0000", 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.9004244640000003, 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20059646, 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.008 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.046379+0000", 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.046379+0000", 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.009 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.046380+0000", 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8899999999999998e-07 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.046395+0000", 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.467e-05 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.010 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.046416+0000", 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1012000000000001e-05 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.146504+0000", 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10008810899999999 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.146600+0000", 2024-04-18T23:11:55.011 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6515999999999995e-05 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.146613+0000", 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2149e-05 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.246937+0000", 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100324151 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.012 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.246976+0000", 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8964000000000002e-05 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 82/72 ECSubWrite(tid=194, reqid=client.4568.0:674, at_version=82'98, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.661633+0000", 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.285170548, 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30563710599999999, 2024-04-18T23:11:55.013 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.661633+0000", 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.014 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.661633+0000", 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.661635+0000", 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.662171+0000", 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053610699999999997 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.015 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.662200+0000", 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8795000000000001e-05 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.662222+0000", 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2362999999999999e-05 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762467+0000", 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100244747 2024-04-18T23:11:55.016 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762499+0000", 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1915000000000002e-05 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.967208+0000", 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20470896599999999 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.017 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.967270+0000", 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2663000000000006e-05 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 82/72 ECSubWrite(tid=195, reqid=client.4568.0:675, at_version=82'99, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.760858+0000", 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.1859453349999995, 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20646499700000001, 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.018 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.760858+0000", 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.760858+0000", 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.019 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.760861+0000", 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5059999999999998e-06 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.761310+0000", 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044913199999999998 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.020 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.761341+0000", 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1362000000000002e-05 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.761357+0000", 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5792999999999999e-05 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.764635+0000", 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032775450000000002 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.021 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.764664+0000", 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9218999999999999e-05 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.967284+0000", 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20261981500000001 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.967323+0000", 2024-04-18T23:11:55.022 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9625000000000001e-05 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 82/72 ECSubWrite(tid=196, reqid=client.4568.0:676, at_version=82'100, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.762672+0000", 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.1841314939999998, 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20774319099999999, 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.023 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762672+0000", 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762672+0000", 2024-04-18T23:11:55.024 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762674+0000", 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1670000000000002e-06 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762829+0000", 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000155081 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.025 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762858+0000", 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8119e-05 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762873+0000", 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5559000000000001e-05 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.026 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.966297+0000", 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20342418200000001 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.966314+0000", 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6609e-05 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.970345+0000", 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040308879999999998 2024-04-18T23:11:55.027 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.970415+0000", 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0586000000000005e-05 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 82/31 ECSubWrite(tid=115, reqid=osd.8.0:115, at_version=82'72, trim_to=0'0, roll_forward_to=81'70))", 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.777459+0000", 2024-04-18T23:11:55.028 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.1693445660000004, 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20149748300000001, 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777459+0000", 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.029 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777459+0000", 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777460+0000", 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9999999999999996e-07 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777511+0000", 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0643000000000003e-05 2024-04-18T23:11:55.030 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777540+0000", 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9419000000000001e-05 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.877642+0000", 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100102052 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.031 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.877807+0000", 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016472399999999999 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.877826+0000", 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8958999999999999e-05 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.978871+0000", 2024-04-18T23:11:55.032 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10104500399999999 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.978957+0000", 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5882000000000002e-05 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 87/72 ECSubWrite(tid=230, reqid=osd.14.0:230, at_version=87'122, trim_to=0'0, roll_forward_to=87'120))", 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:53.983920+0000", 2024-04-18T23:11:55.033 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.96288392499999997, 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201235989, 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983920+0000", 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.034 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983920+0000", 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983921+0000", 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5799999999999999e-06 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983971+0000", 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9509999999999999e-05 2024-04-18T23:11:55.035 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983996+0000", 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4819000000000001e-05 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.084083+0000", 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100087079 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.036 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.084223+0000", 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014055 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.084252+0000", 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8286e-05 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.185099+0000", 2024-04-18T23:11:55.037 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100847145 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.185156+0000", 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.702e-05 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 87/31 ECSubWrite(tid=377, reqid=osd.4.0:374, at_version=0'0, trim_to=0'0, roll_forward_to=87'149))", 2024-04-18T23:11:55.038 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:54.146955+0000", 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.79984843100000003, 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.203462847, 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.146955+0000", 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.039 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.146955+0000", 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.146957+0000", 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7859999999999999e-06 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.040 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.146978+0000", 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0933000000000001e-05 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.147007+0000", 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9288000000000001e-05 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.247164+0000", 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100157156 2024-04-18T23:11:55.041 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.347422+0000", 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10025711900000001 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.347455+0000", 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3575999999999998e-05 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.042 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.350346+0000", 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028906890000000001 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.350418+0000", 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2299999999999996e-05 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.043 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:55.045 INFO:tasks.rados.rados.0.smithi142.stdout:321: finishing write tid 2 to smithi14230322-33 2024-04-18T23:11:55.045 INFO:tasks.rados.rados.0.smithi142.stdout:321: finishing write tid 3 to smithi14230322-33 2024-04-18T23:11:55.045 INFO:tasks.rados.rados.0.smithi142.stdout:321: finishing write tid 4 to smithi14230322-33 2024-04-18T23:11:55.045 INFO:tasks.rados.rados.0.smithi142.stdout:322: finishing copy_from to smithi14230322-18 2024-04-18T23:11:55.045 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 18 v 76 (ObjNum 94 snap 21 seq_num 94) dirty exists 2024-04-18T23:11:55.046 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:55.043+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:55.046 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:11:55.145 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:55.142+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:55.174 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:55.174 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:55.175 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:55.175 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:55.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:11:55.213 INFO:tasks.rados.rados.0.smithi142.stdout:321: finishing write tid 5 to smithi14230322-33 2024-04-18T23:11:55.275 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:55.274+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:55.314 INFO:tasks.rados.rados.0.smithi142.stdout:321: finishing write tid 6 to smithi14230322-33 2024-04-18T23:11:55.347 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:55.347+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:55.355 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:55.355 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:55.355 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:55.355 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:55.355 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:55.369 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:11:55.419 INFO:tasks.rados.rados.0.smithi142.stdout:321: finishing write tid 7 to smithi14230322-33 2024-04-18T23:11:55.420 INFO:tasks.rados.rados.0.smithi142.stdout:321: oid 33 updating version 0 to 114 2024-04-18T23:11:55.420 INFO:tasks.rados.rados.0.smithi142.stdout:321: oid 33 updating version 114 to 117 2024-04-18T23:11:55.420 INFO:tasks.rados.rados.0.smithi142.stdout:321: oid 33 version 117 is already newer than 116 2024-04-18T23:11:55.420 INFO:tasks.rados.rados.0.smithi142.stdout:321: oid 33 version 117 is already newer than 112 2024-04-18T23:11:55.420 INFO:tasks.rados.rados.0.smithi142.stdout:321: oid 33 version 117 is already newer than 115 2024-04-18T23:11:55.420 INFO:tasks.rados.rados.0.smithi142.stdout:321: oid 33 version 117 is already newer than 113 2024-04-18T23:11:55.420 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 33 v 117 (ObjNum 122 snap 28 seq_num 122) dirty exists 2024-04-18T23:11:55.420 INFO:tasks.rados.rados.0.smithi142.stdout:321: left oid 33 (ObjNum 122 snap 28 seq_num 122) 2024-04-18T23:11:55.448 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:55.449+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:176 3.8s0 3:1090b2d1:::smithi14230322-15:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.002266906000003, 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.53110666500000003, 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.544 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 176 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539638+0000", 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3829999999999999e-06 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539651+0000", 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2549e-05 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539675+0000", 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4413000000000002e-05 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539689+0000", 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4161999999999999e-05 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539831+0000", 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014226500000000001 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op must wait for map", 2024-04-18T23:11:55.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539865+0000", 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3185000000000002e-05 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644415+0000", 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10455017599999999 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644498+0000", 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3568999999999999e-05 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.865827+0000", 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.22132859799999999 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.868846+0000", 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030189309999999999 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070576+0000", 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20172967 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070742+0000", 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016676399999999999 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:177 3.8s0 3:1090b2d1:::smithi14230322-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 55.002184274000001, 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32703084799999999, 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.550 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 177 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539719+0000", 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.37e-07 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539723+0000", 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1620000000000001e-06 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539733+0000", 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4630000000000007e-06 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539739+0000", 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7880000000000001e-06 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540029+0000", 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029001100000000001 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540050+0000", 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1033e-05 2024-04-18T23:11:55.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644678+0000", 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10462763999999999 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644714+0000", 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6202999999999999e-05 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866546+0000", 2024-04-18T23:11:55.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.22183165399999999 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866650+0000", 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104465 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866749+0000", 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8791999999999999e-05 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.555 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:207 3.8s0 3:1c19a06b:::smithi14230322-28:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.980568107000003, 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40709379200000001, 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 207 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561336+0000", 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.638e-06 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561354+0000", 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7413000000000001e-05 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561372+0000", 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8003e-05 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561383+0000", 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1425e-05 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561495+0000", 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111661 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561506+0000", 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.113e-05 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644748+0000", 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.083242308000000001 2024-04-18T23:11:55.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644807+0000", 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8909999999999997e-05 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.854271+0000", 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.209463753 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.856288+0000", 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020173909999999999 2024-04-18T23:11:55.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.968263+0000", 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.111975256 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.968428+0000", 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016490400000000001 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.561 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:208 3.8s0 3:1c19a06b:::smithi14230322-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 54.980495017999999, 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29446849200000003, 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 208 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561411+0000", 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9179999999999998e-06 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561421+0000", 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4599999999999992e-06 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561427+0000", 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561582+0000", 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000154652 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561592+0000", 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.004e-05 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644937+0000", 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.083345591999999996 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644974+0000", 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6835000000000002e-05 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855704+0000", 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.21072980599999999 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855787+0000", 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3442000000000005e-05 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855876+0000", 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8660999999999997e-05 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:331 3.8s0 3:1c19a06b:::smithi14230322-28:head [stat,delete] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.113015253999997, 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205262474, 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.567 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 331 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428890+0000", 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4830000000000002e-06 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428901+0000", 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0852999999999999e-05 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428933+0000", 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2234999999999999e-05 2024-04-18T23:11:55.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428956+0000", 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3436000000000001e-05 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.429116+0000", 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015961600000000001 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.429402+0000", 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028557099999999998 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.431719+0000", 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002317117 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.434261+0000", 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025422980000000001 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.633920+0000", 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.199659052 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.634150+0000", 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022981299999999999 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 42.810827678000003, 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30085146299999999, 2024-04-18T23:11:55.572 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731076+0000", 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6599999999999994e-07 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731085+0000", 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6670000000000002e-06 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731102+0000", 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7368999999999999e-05 2024-04-18T23:11:55.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731113+0000", 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1552999999999999e-05 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031861+0000", 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30074756000000002 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031926+0000", 2024-04-18T23:11:55.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5347999999999998e-05 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:556 3.8s0 3:1c19a06b:::smithi14230322-28:head [append 0~0] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.572903+0000", 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.968999750999998, 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5168125699999999, 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.576 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 556 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572903+0000", 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572903+0000", 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572906+0000", 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6970000000000002e-06 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572916+0000", 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0839000000000001e-05 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572947+0000", 2024-04-18T23:11:55.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0433000000000002e-05 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572962+0000", 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5645999999999999e-05 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573159+0000", 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000196949 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573165+0000", 2024-04-18T23:11:55.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7470000000000002e-06 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.877595+0000", 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3044293819999999 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.878153+0000", 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00055816700000000004 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.879453+0000", 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013006300000000001 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886421+0000", 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069672680000000004 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089573+0000", 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20315222899999999 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089715+0000", 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142583 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:557 3.8s0 3:1c19a06b:::smithi14230322-28:head [rollback 14] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.572993+0000", 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.968910046000001, 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7177589360000001, 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.582 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 557 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572993+0000", 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572993+0000", 2024-04-18T23:11:55.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572993+0000", 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9299999999999996e-07 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572999+0000", 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9100000000000002e-06 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573010+0000", 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1129e-05 2024-04-18T23:11:55.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573018+0000", 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9740000000000002e-06 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573204+0000", 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018570899999999999 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573205+0000", 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.057e-06 2024-04-18T23:11:55.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.883595+0000", 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3103893069999999 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.883773+0000", 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017896500000000001 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.885247+0000", 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014739029999999999 2024-04-18T23:11:55.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089301+0000", 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20405347400000001 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.290623+0000", 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201322048 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.290751+0000", 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000128567 2024-04-18T23:11:55.587 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:558 3.8s0 3:1c19a06b:::smithi14230322-28:head [append 0~0] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.573062+0000", 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.968840200999999, 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.818743402, 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.588 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 558 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573062+0000", 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573062+0000", 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573063+0000", 2024-04-18T23:11:55.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2600000000000001e-07 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573067+0000", 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3900000000000002e-06 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573084+0000", 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-05 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573093+0000", 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7430000000000008e-06 2024-04-18T23:11:55.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.638122+0000", 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.065028498000000004 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.638126+0000", 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5530000000000003e-06 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.986937+0000", 2024-04-18T23:11:55.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3488105429999999 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.987167+0000", 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023003800000000001 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.988303+0000", 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011358970000000001 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089734+0000", 2024-04-18T23:11:55.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101431382 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.391689+0000", 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30195531799999997 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.391806+0000", 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000116514 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.593 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 71/70 ECSubWriteReply(tid=32, last_complete=71'22, committed=1, applied=1))", 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.886489+0000", 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.655414012000001, 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202604601, 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886489+0000", 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886489+0000", 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886490+0000", 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7770000000000001e-06 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886501+0000", 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0353000000000001e-05 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886529+0000", 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8343e-05 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886549+0000", 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9726000000000001e-05 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089054+0000", 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20250496000000001 2024-04-18T23:11:55.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089093+0000", 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9441999999999999e-05 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:634 3.8s0 3:1090b2d1:::smithi14230322-15:head [create,setxattr UrPUWbaU]cNfinVtoVUNsYX\\ahn\\oSdRW\\N[RcfO (807) in=40b,setxattr ZQoeVnpeaooblfNZklrg\\hmphZ^pTVh`eRXheade (1225) in=40b,setxattr kcjR][ah`rcmsWjUVlUeaofblgmpleUjjs^]ai\\t (7928) in=40b,setxattr `U^gRRs]UWRVngUaNeN^`p`aVl\\cn_a\\S`iTpOVo (1156) in=40b,setxattr aOYbW^`[gQfVcTQrq[NYeqh\\Q_bjiaWOefngkiNR (1331) in=40b,setxattr igNjm (735) in=5b,setxattr aQYs^c[hafOm^\\g[^R\\^QOd^eiS\\ghimncRcb_pi (15856) in=40b,setxattr dZUpqXmQYdS]obbkNRYdSsdUpenRkjiSSfUmqRTd (9612) in=40b,setxattr lX^flfV^[d^WmlW_UT^kV`s^ahfcNQsT`de`^spi (1612) in=40b,setxattr Ua`UXimY^dWVoQaji\\cZpP^er^hlbYQZs`USR`oh (10033) in=40b,setxattr XhOccedqUYhrPkk[^dVtangb_r^]alTl`ogaOPag (840) in=40b,setxattr YqbXbb\\oPeeR]rqcNaSsS`lkYoffXPcpaqhrb^te (418) in=40b,setxattr jdOeY]PQnfd[iZ^ph\\ijd[WRRnQ^phTpRROYeXss (1611) in=40b,setxattr ie\\ahhoaskiWrasYNh\\mUkqRso\\Olj\\c^P_\\bPp\\ (15577) in=40b,setxattr Ptb\\gnjjhTnd`OQag[dXTYa]t_id[PRpjt_ZfVY] (1541) in=40b,setxattr sSR]a[NPVaUQhPkla[ZVX`lTdim`fgjgeRtm`tt^ (841) in=40b,setxattr _cf[gnNmUkknhenPpY^\\lefOR_n\\mcb`iqOOPUiW (11296) in=40b,setxattr d[TVh[TTre^P`mnfob^WP[dckRkQZafjR[PaW^Zn (8912) in=40b,setxattr X[gWOjQtntoVfbZV[UObOVPoj`qib[[Zt]trgVjh (1542) in=40b,setxattr _qa\\VSbX\\\\b`tO_lSRnsf`PVNfmPkWrf[P\\ZTej[ (453) in=40b] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.865771+0000", 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 11.676131198, 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30787136700000001, 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.597 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 634 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865771+0000", 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865771+0000", 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865774+0000", 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9000000000000002e-06 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865868+0000", 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3826999999999995e-05 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865902+0000", 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3562000000000002e-05 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865918+0000", 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6189e-05 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.966141+0000", 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10022311 2024-04-18T23:11:55.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.966365+0000", 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000223556 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.968971+0000", 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026065469999999999 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.071518+0000", 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102546892 2024-04-18T23:11:55.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.173465+0000", 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10194739799999999 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.173643+0000", 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000177386 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 82/31 ECSubWrite(tid=312, reqid=osd.4.0:308, at_version=0'0, trim_to=0'0, roll_forward_to=81'162))", 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.083084+0000", 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.4588186780000001, 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30191269900000001, 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083084+0000", 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083084+0000", 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083085+0000", 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4160000000000001e-06 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083116+0000", 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0664999999999999e-05 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083146+0000", 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9689e-05 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183319+0000", 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100173805 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183491+0000", 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000171757 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183521+0000", 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9581999999999999e-05 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.384935+0000", 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201414449 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.384997+0000", 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1335999999999997e-05 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.606 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 82/31 ECSubWrite(tid=314, reqid=osd.4.0:314, at_version=82'166, trim_to=0'0, roll_forward_to=81'162))", 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.183484+0000", 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.3584183789999997, 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20244278099999999, 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183484+0000", 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183484+0000", 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183485+0000", 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7999999999999995e-07 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183516+0000", 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1684999999999999e-05 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183529+0000", 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2872e-05 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183539+0000", 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8500000000000006e-06 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.384381+0000", 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.200841872 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.384401+0000", 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9639e-05 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.385858+0000", 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001457243 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.385927+0000", 2024-04-18T23:11:55.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9040000000000003e-05 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:690 3.8s0 3:1090b2d1:::smithi14230322-15:head [append 0~704512] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.807395+0000", 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.7345076219999997, 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.51943692100000005, 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.611 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 690 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.807395+0000", 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.807395+0000", 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.807398+0000", 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.253e-06 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.808935+0000", 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536412 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.808969+0000", 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4051000000000003e-05 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.808985+0000", 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6186000000000001e-05 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.909266+0000", 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100281412 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.909664+0000", 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039748600000000001 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.913002+0000", 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033379619999999999 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.024906+0000", 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.111904329 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.326692+0000", 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30178585099999999 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.326832+0000", 2024-04-18T23:11:55.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013997900000000001 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:691 3.8s0 3:1090b2d1:::smithi14230322-15:head [append 0~655360] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.814196+0000", 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.7277069010000004, 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.61312221899999997, 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.616 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 691 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.814196+0000", 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.814196+0000", 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.814197+0000", 2024-04-18T23:11:55.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4529999999999999e-06 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.815127+0000", 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00092968399999999996 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.815148+0000", 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1671999999999999e-05 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915240+0000", 2024-04-18T23:11:55.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100091456 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915549+0000", 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000308547 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915731+0000", 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018246000000000001 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.917602+0000", 2024-04-18T23:11:55.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001871277 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.125440+0000", 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.207837408 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427190+0000", 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30175016300000002 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427318+0000", 2024-04-18T23:11:55.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012809900000000001 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:692 3.8s0 3:1090b2d1:::smithi14230322-15:head [append 0~507904] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.915310+0000", 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.6265925970000001, 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.51235423099999999, 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.621 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 692 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915310+0000", 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915310+0000", 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915312+0000", 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0870000000000001e-06 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916047+0000", 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00073506099999999996 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916075+0000", 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7657999999999998e-05 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916088+0000", 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3217000000000001e-05 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.919103+0000", 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030152550000000001 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.919395+0000", 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029170500000000001 2024-04-18T23:11:55.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.921531+0000", 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021356930000000001 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.125444+0000", 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20391351699999999 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427541+0000", 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30209633800000002 2024-04-18T23:11:55.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427664+0000", 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001237 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:693 3.8s0 3:1090b2d1:::smithi14230322-15:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.916109+0000", 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.6257934819999997, 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.54009619900000005, 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.626 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 693 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916109+0000", 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916109+0000", 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916110+0000", 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0699999999999999e-06 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916115+0000", 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.242e-06 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916126+0000", 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0414000000000001e-05 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916136+0000", 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0349e-05 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.023380+0000", 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10724339400000001 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.023571+0000", 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019098699999999999 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.024464+0000", 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089320100000000004 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.225628+0000", 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201164015 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.455934+0000", 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.23030629799999999 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.456205+0000", 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000271229 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.631 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:694 3.8s0 3:1090b2d1:::smithi14230322-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.916170+0000", 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.6257329619999998, 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.74335174800000003, 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 694 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916170+0000", 2024-04-18T23:11:55.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916170+0000", 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916171+0000", 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.33e-07 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916176+0000", 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8799999999999996e-06 2024-04-18T23:11:55.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916187+0000", 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0949e-05 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.016296+0000", 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100108797 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.125250+0000", 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10895395200000001 2024-04-18T23:11:55.634 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.125424+0000", 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000174091 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.457293+0000", 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.33186873900000002 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.457553+0000", 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026042900000000001 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.635 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.659521+0000", 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20196797799999999 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 83/70 ECSubWriteReply(tid=60, last_complete=83'53, committed=1, applied=1))", 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.116515+0000", 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.4253880600000004, 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20973929499999999, 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.636 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116515+0000", 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116515+0000", 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116516+0000", 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.469e-06 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.637 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116524+0000", 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4840000000000004e-06 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116551+0000", 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6721999999999999e-05 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116589+0000", 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8268e-05 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.638 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.326219+0000", 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20962908399999999 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.326254+0000", 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5268000000000002e-05 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 83/70 ECSubWriteReply(tid=60, last_complete=83'53, committed=1, applied=1))", 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.172192+0000", 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.3697106540000004, 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.25482231500000002, 2024-04-18T23:11:55.639 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172192+0000", 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172192+0000", 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:55.640 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172193+0000", 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172202+0000", 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.636e-06 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172228+0000", 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5879000000000001e-05 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172241+0000", 2024-04-18T23:11:55.641 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2903e-05 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.426978+0000", 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.25473670199999998 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427014+0000", 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6644999999999999e-05 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:55.642 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:55.643 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:55.644 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:11:55.645 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:55.642+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:55.691 INFO:tasks.rados.rados.0.smithi142.stdout:321: done (2 left) 2024-04-18T23:11:55.691 INFO:tasks.rados.rados.0.smithi142.stdout:322: done (1 left) 2024-04-18T23:11:55.691 INFO:tasks.rados.rados.0.smithi142.stdout:323: done (0 left) 2024-04-18T23:11:55.691 INFO:tasks.rados.rados.0.smithi142.stdout:324: snap_create 2024-04-18T23:11:55.744 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:55.742+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:55.815 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:55.815 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:55.815 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:55.815 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:55.829 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:11:55.845 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:55.842+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:55.945 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:55.942+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:55.990 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:55.990 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:55.991 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:55.991 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:55.991 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:56.004 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:11:56.046 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:11:56.042+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:11:56.147 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:56.147+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:56.174 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:56.174 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:56.174 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 57/31 ECSubWrite(tid=167, reqid=osd.4.0:167, at_version=57'77, trim_to=0'0, roll_forward_to=56'73))", 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:22.631249+0000", 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.543639888000001, 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20201448899999999, 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.175 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631249+0000", 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631249+0000", 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631250+0000", 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0559999999999999e-06 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.176 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631287+0000", 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7069e-05 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631308+0000", 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0723000000000001e-05 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.731401+0000", 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100093711 2024-04-18T23:11:56.177 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.831671+0000", 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100269671 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.831687+0000", 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6283999999999999e-05 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.833165+0000", 2024-04-18T23:11:56.178 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014771879999999999 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.833263+0000", 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8787000000000001e-05 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 57/31 ECSubWrite(tid=170, reqid=osd.4.0:167, at_version=0'0, trim_to=0'0, roll_forward_to=57'77))", 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:22.834221+0000", 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.340667605999997, 2024-04-18T23:11:56.179 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30088188700000001, 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834221+0000", 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834221+0000", 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.180 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834223+0000", 2024-04-18T23:11:56.181 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.516e-06 2024-04-18T23:11:56.181 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.181 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.181 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.181 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834241+0000", 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8712999999999999e-05 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834271+0000", 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9963000000000001e-05 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.934412+0000", 2024-04-18T23:11:56.182 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10014097499999999 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.934586+0000", 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000173535 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.934601+0000", 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5574999999999999e-05 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.183 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:23.135038+0000", 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20043703500000001 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:23.135103+0000", 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4574999999999998e-05 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 61/31 ECSubWrite(tid=190, reqid=osd.4.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.458871+0000", 2024-04-18T23:11:56.184 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.716018105, 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20139204499999999, 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458871+0000", 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458871+0000", 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.185 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458872+0000", 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5740000000000001e-06 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458892+0000", 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0060000000000001e-05 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458921+0000", 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8714999999999999e-05 2024-04-18T23:11:56.186 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458936+0000", 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5296000000000001e-05 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.559175+0000", 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100238271 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.559198+0000", 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3133e-05 2024-04-18T23:11:56.187 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.660126+0000", 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100928437 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.660263+0000", 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013655899999999999 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.188 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 62/31 ECSubWrite(tid=191, reqid=client.4568.0:474, at_version=62'89, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.579357+0000", 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.595531472000001, 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205588628, 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579357+0000", 2024-04-18T23:11:56.189 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579357+0000", 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579358+0000", 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.190 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.580046+0000", 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068822000000000002 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.580081+0000", 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4882000000000002e-05 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680226+0000", 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100145083 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.191 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680407+0000", 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000181068 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680437+0000", 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9983000000000001e-05 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.784862+0000", 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.104425411 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.192 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.784946+0000", 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3608000000000003e-05 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 70/31 ECSubWrite(tid=60, reqid=client.4568.0:554, at_version=70'46, trim_to=0'0, roll_forward_to=70'44))", 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.879929+0000", 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 19.294959989999999, 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20180873899999999, 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.193 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.879929+0000", 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.879929+0000", 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.194 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.879931+0000", 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.879974+0000", 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3755999999999999e-05 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.880012+0000", 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7276e-05 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.195 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.980154+0000", 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100141984 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.080426+0000", 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100272718 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.080443+0000", 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6382999999999999e-05 2024-04-18T23:11:56.196 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.081661+0000", 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001218418 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.081737+0000", 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6248000000000006e-05 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.197 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.6s6 70/68 ECSubRead(tid=36, to_read={3:635103ff:::smithi14230322-31:head=0,262144,0}, subchunks={3:635103ff:::smithi14230322-31:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.390943+0000", 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.783945211999999, 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30072780199999999, 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390943+0000", 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.198 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390943+0000", 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390945+0000", 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390963+0000", 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7180000000000002e-05 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.199 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390997+0000", 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4616000000000002e-05 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.491149+0000", 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100151937 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.691524+0000", 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20037495599999999 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.200 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.691671+0000", 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147233 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.as3 71/31 ECSubWrite(tid=19, reqid=osd.3.0:19, at_version=71'8, trim_to=0'0, roll_forward_to=69'6))", 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.577755+0000", 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 18.597133252999999, 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20409171300000001, 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.201 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577755+0000", 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577755+0000", 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577756+0000", 2024-04-18T23:11:56.202 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577795+0000", 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9125000000000002e-05 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577818+0000", 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3368999999999999e-05 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.677913+0000", 2024-04-18T23:11:56.203 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10009472799999999 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.778121+0000", 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10020746799999999 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.778137+0000", 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6852999999999999e-05 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.204 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.781737+0000", 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035999230000000001 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.781847+0000", 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109883 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 72/31 ECSubWrite(tid=66, reqid=osd.2.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-04-18T23:11:56.205 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:38.620535+0000", 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.554354054000001, 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20162907699999999, 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620535+0000", 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.206 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620535+0000", 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620535+0000", 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2600000000000001e-07 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620554+0000", 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8617000000000001e-05 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620566+0000", 2024-04-18T23:11:56.207 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1931e-05 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.720651+0000", 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100085153 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.820810+0000", 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100159237 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.208 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.820822+0000", 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1331999999999999e-05 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.822128+0000", 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001305816 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.822164+0000", 2024-04-18T23:11:56.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6164999999999997e-05 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 72/31 ECSubWrite(tid=214, reqid=osd.4.0:214, at_version=72'104, trim_to=0'0, roll_forward_to=66'102))", 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:38.716533+0000", 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.458355309000002, 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202635437, 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716533+0000", 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716533+0000", 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716534+0000", 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8099999999999999e-07 2024-04-18T23:11:56.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716557+0000", 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3773000000000001e-05 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716578+0000", 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0633999999999999e-05 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716589+0000", 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-05 2024-04-18T23:11:56.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716738+0000", 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014939799999999999 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716756+0000", 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7802999999999999e-05 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.919099+0000", 2024-04-18T23:11:56.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20234323300000001 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.919169+0000", 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9665000000000005e-05 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 72/31 ECSubWrite(tid=216, reqid=osd.4.0:216, at_version=72'106, trim_to=0'0, roll_forward_to=66'102))", 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:38.717905+0000", 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.456983479000002, 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.301742538, 2024-04-18T23:11:56.214 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.717905+0000", 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.717905+0000", 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.717907+0000", 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.57e-06 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.718037+0000", 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012998299999999999 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.718069+0000", 2024-04-18T23:11:56.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1890000000000001e-05 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.718095+0000", 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6072e-05 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.918419+0000", 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20032456900000001 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.217 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.918440+0000", 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0632000000000002e-05 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.019609+0000", 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101169125 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.019648+0000", 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8696999999999999e-05 2024-04-18T23:11:56.218 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 72/31 ECSubWrite(tid=67, reqid=osd.2.0:66, at_version=0'0, trim_to=0'0, roll_forward_to=72'52))", 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:38.824136+0000", 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.350753078, 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20173307800000001, 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824136+0000", 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824136+0000", 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824137+0000", 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4799999999999997e-07 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824148+0000", 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1686e-05 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824163+0000", 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4545000000000001e-05 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824175+0000", 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2678e-05 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.024455+0000", 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20027978799999999 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.024472+0000", 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6782999999999998e-05 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.025834+0000", 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013615999999999999 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.025869+0000", 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5049999999999998e-05 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 77/31 ECSubWrite(tid=246, reqid=client.4568.0:606, at_version=77'84, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.901108+0000", 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.27378055, 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20364768699999999, 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.223 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901108+0000", 2024-04-18T23:11:56.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901108+0000", 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901110+0000", 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6700000000000001e-06 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901608+0000", 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049785200000000002 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901633+0000", 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5055999999999999e-05 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901651+0000", 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8428e-05 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.001855+0000", 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100203588 2024-04-18T23:11:56.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.001877+0000", 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1971999999999999e-05 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.104674+0000", 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102796919 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.104756+0000", 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2201999999999997e-05 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 77/31 ECSubWrite(tid=247, reqid=client.4568.0:607, at_version=77'85, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.901685+0000", 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.273203616, 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20315154799999999, 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.228 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901685+0000", 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901685+0000", 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901686+0000", 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8700000000000004e-07 2024-04-18T23:11:56.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901908+0000", 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221843 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901926+0000", 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7932e-05 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.002023+0000", 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100097214 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.102305+0000", 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10028247699999999 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.102326+0000", 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0879000000000001e-05 2024-04-18T23:11:56.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.104773+0000", 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024464159999999999 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.104837+0000", 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3899999999999995e-05 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.232 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 82/31 ECSubWrite(tid=311, reqid=osd.4.0:311, at_version=82'124, trim_to=0'0, roll_forward_to=81'120))", 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.955944+0000", 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 7.2189444749999998, 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202757677, 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.955944+0000", 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.955944+0000", 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.955946+0000", 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.793e-06 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.955988+0000", 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1681999999999998e-05 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.956017+0000", 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9077999999999998e-05 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.056165+0000", 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100147999 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.156449+0000", 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10028411499999999 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.156481+0000", 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2094999999999997e-05 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.158639+0000", 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021584590000000002 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.158702+0000", 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2456e-05 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 83/31 ECSubWrite(tid=318, reqid=client.4568.0:695, at_version=83'126, trim_to=0'0, roll_forward_to=82'124))", 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.830934+0000", 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.3439550479999998, 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20389481700000001, 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.237 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.830934+0000", 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.830934+0000", 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.830936+0000", 2024-04-18T23:11:56.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.12e-06 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.831373+0000", 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043691699999999999 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.831407+0000", 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4771000000000003e-05 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.931510+0000", 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010235200000001 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.031835+0000", 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10032532 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.031872+0000", 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6514999999999999e-05 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.034733+0000", 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00286141 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.034828+0000", 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5412000000000001e-05 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.6s6 83/72 ECSubWrite(tid=53, reqid=client.4568.0:724, at_version=83'40, trim_to=0'0, roll_forward_to=81'36))", 2024-04-18T23:11:56.241 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.991435+0000", 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.1834537589999998, 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201353955, 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991435+0000", 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991435+0000", 2024-04-18T23:11:56.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991436+0000", 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3e-07 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991452+0000", 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6852000000000001e-05 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991464+0000", 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1362000000000001e-05 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991473+0000", 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3370000000000006e-06 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.191528+0000", 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20005491 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.191557+0000", 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9108999999999999e-05 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.192743+0000", 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011859939999999999 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.192789+0000", 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5760999999999998e-05 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.245 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 84/31 ECSubWrite(tid=337, reqid=client.4568.0:754, at_version=84'177, trim_to=0'0, roll_forward_to=83'173))", 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.964705+0000", 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 5.21018369, 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20257407599999999, 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964705+0000", 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964705+0000", 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964707+0000", 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8929999999999999e-06 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.247 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964903+0000", 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000195873 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964934+0000", 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0972000000000001e-05 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.065082+0000", 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100148376 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.065242+0000", 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015953199999999999 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.065270+0000", 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7915e-05 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.167201+0000", 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10193097800000001 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.167279+0000", 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8536999999999997e-05 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 87/31 ECSubWrite(tid=380, reqid=osd.4.0:376, at_version=0'0, trim_to=0'0, roll_forward_to=87'198))", 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:54.371773+0000", 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.803115477, 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20143276099999999, 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.250 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371773+0000", 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371773+0000", 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371775+0000", 2024-04-18T23:11:56.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0200000000000001e-06 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371797+0000", 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2146000000000001e-05 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371827+0000", 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0071999999999999e-05 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371844+0000", 2024-04-18T23:11:56.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6849000000000001e-05 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.472133+0000", 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100288628 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.472150+0000", 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7241999999999999e-05 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573153+0000", 2024-04-18T23:11:56.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101003211 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573206+0000", 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2593000000000003e-05 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 87/31 ECSubWrite(tid=381, reqid=osd.4.0:381, at_version=87'200, trim_to=0'0, roll_forward_to=87'198))", 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:54.371887+0000", 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.8030016720000002, 2024-04-18T23:11:56.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202086396, 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371887+0000", 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371887+0000", 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371888+0000", 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9699999999999996e-07 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371922+0000", 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4378000000000002e-05 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371939+0000", 2024-04-18T23:11:56.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7173e-05 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371950+0000", 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0572e-05 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.572712+0000", 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.200762725 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.572722+0000", 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0012e-05 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573932+0000", 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012091770000000001 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573973+0000", 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1862e-05 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.258 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 87/31 ECSubWrite(tid=382, reqid=osd.4.0:382, at_version=87'202, trim_to=0'0, roll_forward_to=87'198))", 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:54.371970+0000", 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 1.802918456, 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202436489, 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371970+0000", 2024-04-18T23:11:56.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371970+0000", 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371971+0000", 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6699999999999999e-07 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.372005+0000", 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4375000000000002e-05 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.372017+0000", 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1429e-05 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.372026+0000", 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2669999999999995e-06 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573285+0000", 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20125895999999999 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573299+0000", 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4156e-05 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.574348+0000", 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001049358 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.574407+0000", 2024-04-18T23:11:56.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8477000000000003e-05 2024-04-18T23:11:56.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.263 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.263 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.263 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:56.265 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:11:56.265 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:56.265+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:56.366 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:11:56.362+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:11:56.377 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:11:56.391 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:56.391 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:56.391 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:11:56.391 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:56.406 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:11:56.467 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:56.462+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:56.533 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:56.533 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:56.553 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:56.554 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:56.567 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:56.562+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:56.572 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:56.572 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:11:56.572 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:11:56.572 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:11:56.572 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:56.575 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:56.575 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:56.586 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:11:56.668 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:56.669+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:56.689 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:56.689 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:56.691 INFO:tasks.rados.rados.0.smithi142.stdout:324: done (0 left) 2024-04-18T23:11:56.691 INFO:tasks.rados.rados.0.smithi142.stdout:325: read oid 15 snap -1 2024-04-18T23:11:56.691 INFO:tasks.rados.rados.0.smithi142.stdout:325: expect (ObjNum 111 snap 26 seq_num 111) 2024-04-18T23:11:56.691 INFO:tasks.rados.rados.0.smithi142.stdout:326: snap_create 2024-04-18T23:11:56.699 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T23:11:56.707 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:11:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:11:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:11:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:11:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.112515528000003, 2024-04-18T23:11:56.707 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62069844100000005, 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 114 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.708 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594565+0000", 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594570+0000", 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.186e-06 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.709 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594582+0000", 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1763000000000001e-05 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594590+0000", 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5290000000000003e-06 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010572+0000", 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.415982252 2024-04-18T23:11:56.710 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010727+0000", 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015558400000000001 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010971+0000", 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000243339 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015929+0000", 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049583639999999998 2024-04-18T23:11:56.711 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215108+0000", 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19917934400000001 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215199+0000", 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0556000000000005e-05 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215263+0000", 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3874000000000005e-05 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.712 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.112479200999999, 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62089894099999998, 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:56.713 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 115 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.714 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594746+0000", 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014446700000000001 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594761+0000", 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4792e-05 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594769+0000", 2024-04-18T23:11:56.715 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6160000000000002e-06 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013742+0000", 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.418972438 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013974+0000", 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023265699999999999 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.014328+0000", 2024-04-18T23:11:56.716 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035345 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018069+0000", 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037411079999999999 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215346+0000", 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19727757900000001 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215435+0000", 2024-04-18T23:11:56.717 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8245999999999995e-05 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215500+0000", 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4968000000000005e-05 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.112295950000004, 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.72133309899999998, 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.718 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 116 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.719 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594785+0000", 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594790+0000", 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4229999999999996e-06 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594803+0000", 2024-04-18T23:11:56.720 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2196e-05 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594810+0000", 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6190000000000002e-06 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016062+0000", 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42125235 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016215+0000", 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000152051 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.721 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016420+0000", 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000205916 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018082+0000", 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001661718 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.315968+0000", 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29788616600000001 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.722 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316051+0000", 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2897000000000003e-05 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316117+0000", 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5866999999999998e-05 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:56.723 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 69.112258316999998, 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.721588585, 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 117 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.724 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594829+0000", 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7830000000000001e-06 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.725 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594841+0000", 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1505000000000001e-05 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.694940+0000", 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100099585 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016967+0000", 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32202692799999999 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017224+0000", 2024-04-18T23:11:56.726 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000256548 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017583+0000", 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000359689 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.020139+0000", 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025556649999999999 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:11:56.727 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316256+0000", 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29611688800000002 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316345+0000", 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9285999999999999e-05 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316410+0000", 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5058e-05 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.728 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:26 3.2s0 3:4250b374:::smithi14230322-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.182872291000002, 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60852922899999995, 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 26 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.729 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524210+0000", 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524220+0000", 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0056e-05 2024-04-18T23:11:56.730 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524242+0000", 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2212e-05 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525734+0000", 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001491783 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528099+0000", 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023654380000000001 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.731 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528318+0000", 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000219065 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.529513+0000", 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011952530000000001 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636414+0000", 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106900906 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.732 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132457+0000", 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49604301899999997 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132737+0000", 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027950099999999998 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:31 3.2s0 3:42c34e84:::smithi14230322-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.180757866999997, 2024-04-18T23:11:56.733 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60793644899999999, 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 31 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.734 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526324+0000", 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9400000000000001e-06 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526836+0000", 2024-04-18T23:11:56.735 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051192200000000005 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526869+0000", 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2978000000000002e-05 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526890+0000", 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481999999999999e-05 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531167+0000", 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042762490000000002 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.736 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531508+0000", 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034109400000000001 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.533620+0000", 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021122279999999998 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636418+0000", 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102798509 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.737 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134075+0000", 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.497656453 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134259+0000", 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000183594 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.738 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:32 3.2s0 3:42c34e84:::smithi14230322-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.180154956999999, 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60874687999999999, 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 32 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:56.739 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526927+0000", 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0389999999999999e-06 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526936+0000", 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2599999999999994e-06 2024-04-18T23:11:56.740 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0302e-05 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526973+0000", 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6359e-05 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634473+0000", 2024-04-18T23:11:56.741 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.107499583 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634639+0000", 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016687200000000001 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.635580+0000", 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094034999999999995 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.742 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636830+0000", 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012506220000000001 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135509+0000", 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49867835399999999 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135672+0000", 2024-04-18T23:11:56.743 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000163139 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:618 3.2s0 3:42c34e84:::smithi14230322-11:head [read 0~1835008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 13.084836040000001, 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.63639373399999999, 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.744 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 618 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:11:56.745 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622245+0000", 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622253+0000", 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4889999999999998e-06 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.746 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622276+0000", 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2938999999999998e-05 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722418+0000", 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100141811 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822632+0000", 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100214391 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822645+0000", 2024-04-18T23:11:56.747 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2863e-05 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.841571+0000", 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018926235 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.841706+0000", 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000134528 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.748 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.258638+0000", 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41693209799999997 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:619 3.2s0 3:42c34e84:::smithi14230322-11:head [read 0~1835008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.98462048, 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.94106379600000001, 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.749 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 619 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.750 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722462+0000", 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.12e-06 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722475+0000", 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3475999999999999e-05 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722500+0000", 2024-04-18T23:11:56.751 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4817999999999999e-05 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722517+0000", 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7452000000000002e-05 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822716+0000", 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100198302 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:56.752 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822729+0000", 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3036e-05 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.842152+0000", 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019423458000000001 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.842286+0000", 2024-04-18T23:11:56.753 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013423100000000001 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.663523+0000", 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.82123690299999996 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:628 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~589824] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.867965089, 2024-04-18T23:11:56.754 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.82267251500000005, 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 628 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.755 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839118+0000", 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6759999999999999e-06 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840018+0000", 2024-04-18T23:11:56.756 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090027100000000001 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840051+0000", 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3426999999999997e-05 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840069+0000", 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7869000000000001e-05 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.949596+0000", 2024-04-18T23:11:56.757 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10952640800000001 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.949748+0000", 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001526 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.951307+0000", 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001558633 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.052856+0000", 2024-04-18T23:11:56.758 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101548762 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.661580+0000", 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.608724088 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.661787+0000", 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020778100000000001 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.759 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:629 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~540672] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.860335588, 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91730556200000002, 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 629 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:11:56.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846746+0000", 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5799999999999999e-06 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847461+0000", 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071526900000000004 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847485+0000", 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3509000000000001e-05 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847497+0000", 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2675e-05 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.047393+0000", 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19989558700000001 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.047586+0000", 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000192568 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.049081+0000", 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001495225 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.052942+0000", 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038612070000000002 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.763898+0000", 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.71095591800000002 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764050+0000", 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015202400000000001 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:630 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~622592] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.851473603000001, 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.908871761, 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.765 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 630 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855609+0000", 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.17e-06 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856431+0000", 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082242899999999996 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856463+0000", 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2320999999999998e-05 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856479+0000", 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5396e-05 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.050092+0000", 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19361344699999999 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.050246+0000", 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000153497 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.051908+0000", 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001662562 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.154762+0000", 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102854035 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764322+0000", 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60956000700000001 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764478+0000", 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000155897 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:631 3.2s0 3:4b92d38e:::smithi14230322-19:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:56.770 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.850573935, 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90835886899999996, 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 631 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856507+0000", 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1000000000000001e-06 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856512+0000", 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2290000000000002e-06 2024-04-18T23:11:56.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856534+0000", 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1356e-05 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856543+0000", 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8060000000000001e-06 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.053052+0000", 2024-04-18T23:11:56.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19650978299999999 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.053209+0000", 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015716999999999999 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054047+0000", 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000837925 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.154833+0000", 2024-04-18T23:11:56.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100785849 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764707+0000", 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60987347400000003 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764865+0000", 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015817699999999999 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:632 3.2s0 3:4b92d38e:::smithi14230322-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 12.850524432, 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.212644217, 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 632 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0699999999999996e-07 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856560+0000", 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4230000000000001e-06 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856573+0000", 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3009e-05 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856581+0000", 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0669999999999992e-06 2024-04-18T23:11:56.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054652+0000", 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198071467 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054999+0000", 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034629799999999998 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.766030+0000", 2024-04-18T23:11:56.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.71103122399999996 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.766186+0000", 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015654300000000001 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:45.069200+0000", 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30301327900000002 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:670 3.2s0 3:4250b374:::smithi14230322-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.664265+0000", 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 8.0428152819999994, 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62706280400000003, 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 670 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664265+0000", 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664265+0000", 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664267+0000", 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2239999999999998e-06 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664279+0000", 2024-04-18T23:11:56.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1919000000000001e-05 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664310+0000", 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0855000000000002e-05 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664331+0000", 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1614999999999999e-05 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664485+0000", 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015335899999999999 2024-04-18T23:11:56.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664494+0000", 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4660000000000005e-06 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.088025+0000", 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.423531251 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.088204+0000", 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017829 2024-04-18T23:11:56.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.291327+0000", 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20312382500000001 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:741 3.2s0 3:42341748:::smithi14230322-4:head [append 0~425984] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.968667+0000", 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.7384130029999998, 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91216083299999995, 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.785 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 741 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.968667+0000", 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.968667+0000", 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.968670+0000", 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8559999999999998e-06 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.969441+0000", 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077078300000000004 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.969477+0000", 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6038000000000003e-05 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069573+0000", 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10009589100000001 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070763+0000", 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011900960000000001 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071013+0000", 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025074299999999999 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.073185+0000", 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002171849 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.270509+0000", 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19732339800000001 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.880686+0000", 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.61017750400000004 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.880828+0000", 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014167499999999999 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:742 3.2s0 3:42341748:::smithi14230322-4:head [append 0~606208] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:56.790 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.069661+0000", 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.63741877, 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.81171484500000002, 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 742 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069661+0000", 2024-04-18T23:11:56.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069661+0000", 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069664+0000", 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8930000000000001e-06 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070530+0000", 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086640600000000001 2024-04-18T23:11:56.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070560+0000", 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9538000000000001e-05 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070581+0000", 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0483e-05 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.074800+0000", 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042195740000000002 2024-04-18T23:11:56.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.075058+0000", 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025823000000000003 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.077007+0000", 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019490169999999999 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278274+0000", 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201266476 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881251+0000", 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60297683199999996 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881376+0000", 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125396 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.795 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:743 3.2s0 3:42341748:::smithi14230322-4:head [append 0~540672] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.070618+0000", 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6364617959999999, 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.81111058300000005, 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 743 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070618+0000", 2024-04-18T23:11:56.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070618+0000", 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070620+0000", 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9640000000000002e-06 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071703+0000", 2024-04-18T23:11:56.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001083044 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071733+0000", 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9699000000000001e-05 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071752+0000", 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9275000000000001e-05 2024-04-18T23:11:56.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.270584+0000", 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19883204800000001 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.270766+0000", 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018226000000000001 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.272279+0000", 2024-04-18T23:11:56.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015124030000000001 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.279437+0000", 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071585529999999998 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881609+0000", 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60217155700000002 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881729+0000", 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011977999999999999 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:744 3.2s0 3:42341748:::smithi14230322-4:head [append 0~409600] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.071789+0000", 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.6352913820000001, 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91037796400000004, 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.801 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 744 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071789+0000", 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071789+0000", 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071791+0000", 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1459999999999999e-06 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072442+0000", 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065123700000000004 2024-04-18T23:11:56.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072461+0000", 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9485999999999999e-05 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072479+0000", 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7892000000000001e-05 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.273674+0000", 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20119500400000001 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.273959+0000", 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028440799999999999 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.277315+0000", 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003356412 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.281415+0000", 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041003360000000004 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982028+0000", 2024-04-18T23:11:56.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.70061242700000004 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982167+0000", 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013861599999999999 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:745 3.2s0 3:42341748:::smithi14230322-4:head [setxattr _header (56) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.072506+0000", 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 6.634573799, 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91007206600000001, 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:11:56.806 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 745 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072506+0000", 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072506+0000", 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:11:56.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072508+0000", 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4470000000000001e-06 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072523+0000", 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5756e-05 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072543+0000", 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9712e-05 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072559+0000", 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5758999999999999e-05 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278314+0000", 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205755145 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278426+0000", 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011250499999999999 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278921+0000", 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049428099999999995 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.281726+0000", 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028056639999999998 2024-04-18T23:11:56.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982417+0000", 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.70069052600000004 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982578+0000", 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016127099999999999 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:11:56.811 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:11:56.813 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:56.813 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:56.814 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:56.814 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:56.815 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:11:56.815 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:56.810+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:56.815 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:56.816 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:56.816 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:56.816 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:56.817 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:56.817 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:56.817 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:56.840 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:56.841 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:56.861 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:56.862 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:56.878 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:56.878 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:56.900 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:56.900 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:56.913 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:56.913+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:56.922 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:56.922 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:56.949 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:56.950 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:56.963 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:11:57.014 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:11:57.010+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:11:57.026 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:57.027 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:57.115 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:57.110+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:57.133 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:57.133 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:57.147 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:11:57.215 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:57.217+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:57.267 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:57.268 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:57.281 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:11:57.309 INFO:tasks.rados.rados.0.smithi142.stdout:325: expect (ObjNum 111 snap 26 seq_num 111) 2024-04-18T23:11:57.317 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:57.315+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:57.418 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:57.414+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:57.446 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:57.446 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:57.461 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:11:57.519 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:11:57.519+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:11:57.578 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:57.578 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:57.592 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:11:57.639 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:57.641+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:57.695 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-04-18T23:11:57.696 INFO:tasks.rados.rados.0.smithi142.stdout:325: done (1 left) 2024-04-18T23:11:57.696 INFO:tasks.rados.rados.0.smithi142.stdout:326: done (0 left) 2024-04-18T23:11:57.696 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 23 current snap is 31 2024-04-18T23:11:57.696 INFO:tasks.rados.rados.0.smithi142.stdout:327: seq_num 123 ranges {4210688=524288,4734976=573440,5308416=688128,5996544=294912} 2024-04-18T23:11:57.712 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:57.712 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:57.720 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:11:57.721+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:11:57.727 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:11:57.765 INFO:tasks.rados.rados.0.smithi142.stdout:327: writing smithi14230322-23 from 4210688 to 4734976 tid 1 2024-04-18T23:11:57.771 INFO:tasks.rados.rados.0.smithi142.stdout:327: writing smithi14230322-23 from 4734976 to 5308416 tid 2 2024-04-18T23:11:57.777 INFO:tasks.rados.rados.0.smithi142.stdout:327: writing smithi14230322-23 from 5308416 to 5996544 tid 3 2024-04-18T23:11:57.780 INFO:tasks.rados.rados.0.smithi142.stdout:327: writing smithi14230322-23 from 5996544 to 6291456 tid 4 2024-04-18T23:11:57.780 INFO:tasks.rados.rados.0.smithi142.stdout:328: rollback oid 9 current snap is 31 2024-04-18T23:11:57.780 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 9 to 19 2024-04-18T23:11:57.780 INFO:tasks.rados.rados.0.smithi142.stdout:329: snap_remove snap 28 2024-04-18T23:11:57.821 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:57.822+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:57.899 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:57.899 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:57.913 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:11:57.922 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:57.918+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:58.023 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:58.025+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:58.034 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:58.034 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:58.048 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:11:58.124 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:11:58.123+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:11:58.213 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:58.213 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:58.225 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:58.226+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:58.227 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:11:58.326 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:58.322+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:58.345 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:58.345 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:58.359 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:11:58.426 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:11:58.427+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:11:58.480 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:58.480 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:58.494 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:11:58.510 INFO:tasks.rados.rados.0.smithi142.stdout:329: done (2 left) 2024-04-18T23:11:58.510 INFO:tasks.rados.rados.0.smithi142.stdout:330: rollback oid 25 current snap is 31 2024-04-18T23:11:58.510 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 25 to 29 2024-04-18T23:11:58.510 INFO:tasks.rados.rados.0.smithi142.stdout:328: finishing rollback tid 0 to smithi14230322-9 2024-04-18T23:11:58.510 INFO:tasks.rados.rados.0.smithi142.stdout:327: finishing write tid 1 to smithi14230322-23 2024-04-18T23:11:58.510 INFO:tasks.rados.rados.0.smithi142.stdout:327: finishing write tid 2 to smithi14230322-23 2024-04-18T23:11:58.510 INFO:tasks.rados.rados.0.smithi142.stdout:327: finishing write tid 3 to smithi14230322-23 2024-04-18T23:11:58.510 INFO:tasks.rados.rados.0.smithi142.stdout:328: finishing rollback tid 1 to smithi14230322-9 2024-04-18T23:11:58.510 INFO:tasks.rados.rados.0.smithi142.stdout:327: finishing write tid 4 to smithi14230322-23 2024-04-18T23:11:58.511 INFO:tasks.rados.rados.0.smithi142.stdout:327: finishing write tid 5 to smithi14230322-23 2024-04-18T23:11:58.511 INFO:tasks.rados.rados.0.smithi142.stdout:331: snap_create 2024-04-18T23:11:58.527 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:11:58.526+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:58.620 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:58.628 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:58.628+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:58.635 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:11:58.640 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-18T23:11:58.640 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-18T23:11:58.640 INFO:tasks.thrashosds.thrasher:Shrinking pool unique_pool_0 2024-04-18T23:11:58.640 INFO:tasks.ceph.ceph_manager.ceph:contract_pool unique_pool_0 by 10 min 16 2024-04-18T23:11:58.640 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:11:58.703 INFO:tasks.rados.rados.0.smithi142.stdout:328: finishing rollback tid 2 to smithi14230322-9 2024-04-18T23:11:58.703 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 9 v 81 (ObjNum 70 snap 8 seq_num 70) dirty exists 2024-04-18T23:11:58.728 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:58.726+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:58.765 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:58.766 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:58.780 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:11:58.795 INFO:tasks.rados.rados.0.smithi142.stdout:327: finishing write tid 6 to smithi14230322-23 2024-04-18T23:11:58.795 INFO:tasks.rados.rados.0.smithi142.stdout:327: oid 23 updating version 0 to 161 2024-04-18T23:11:58.795 INFO:tasks.rados.rados.0.smithi142.stdout:327: oid 23 version 161 is already newer than 159 2024-04-18T23:11:58.795 INFO:tasks.rados.rados.0.smithi142.stdout:327: oid 23 version 161 is already newer than 160 2024-04-18T23:11:58.795 INFO:tasks.rados.rados.0.smithi142.stdout:327: oid 23 version 161 is already newer than 158 2024-04-18T23:11:58.795 INFO:tasks.rados.rados.0.smithi142.stdout:327: oid 23 version 161 is already newer than 157 2024-04-18T23:11:58.795 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 23 v 161 (ObjNum 123 snap 31 seq_num 123) dirty exists 2024-04-18T23:11:58.795 INFO:tasks.rados.rados.0.smithi142.stdout:327: left oid 23 (ObjNum 123 snap 31 seq_num 123) 2024-04-18T23:11:58.829 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:11:58.828+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:11:58.904 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:58.904 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:58.919 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:11:58.924 INFO:tasks.rados.rados.0.smithi142.stdout:330: finishing rollback tid 0 to smithi14230322-25 2024-04-18T23:11:58.924 INFO:tasks.rados.rados.0.smithi142.stdout:330: finishing rollback tid 1 to smithi14230322-25 2024-04-18T23:11:58.924 INFO:tasks.rados.rados.0.smithi142.stdout:330: finishing rollback tid 2 to smithi14230322-25 2024-04-18T23:11:58.924 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 25 v 121 (ObjNum 56 snap 4 seq_num 56) dirty exists 2024-04-18T23:11:58.930 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:58.926+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:58.950 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:11:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:11:58.968 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-04-18T23:11:58.508971+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":97583648,"num_objects":105,"num_object_clones":53,"num_object_copies":725,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":5,"num_read":536,"num_read_kb":96848,"num_write":934,"num_write_kb":118372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1434,"ondisk_log_size":1434,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":670724,"kb_used_data":186892,"kb_used_omap":35,"kb_used_meta":483548,"kb_avail":1509278716,"statfs":{"total":1546188226560,"available":1545501405184,"internally_reserved":0,"allocated":191377408,"data_stored":241315383,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":495153793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1,1,0,9],"upper_bound":512},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3440640,"num_objects":-11,"num_object_clones":-11,"num_object_copies":-77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-11,"num_whiteouts":-1,"num_read":6,"num_read_kb":5,"num_write":57,"num_write_kb":10560,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.840762"},"pg_stats":[{"pgid":"3.f","version":"80'19","reported_seq":291,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:54.778507+0000","last_change":"2024-04-18T23:11:53.781951+0000","last_active":"2024-04-18T23:11:54.778507+0000","last_peered":"2024-04-18T23:11:54.778507+0000","last_clean":"2024-04-18T23:11:54.778507+0000","last_became_active":"2024-04-18T23:10:54.696778+0000","last_became_peered":"2024-04-18T23:10:54.696778+0000","last_unstale":"2024-04-18T23:11:54.778507+0000","last_undegraded":"2024-04-18T23:11:54.778507+0000","last_fullsized":"2024-04-18T23:11:54.778507+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:15.976868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10139469399999999,"stat_sum":{"num_bytes":1933312,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":13445,"num_write":6,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.e","version":"87'84","reported_seq":350,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:54.672551+0000","last_change":"2024-04-18T23:11:53.988323+0000","last_active":"2024-04-18T23:11:54.672551+0000","last_peered":"2024-04-18T23:11:54.672551+0000","last_clean":"2024-04-18T23:11:54.672551+0000","last_became_active":"2024-04-18T23:10:54.918517+0000","last_became_peered":"2024-04-18T23:10:54.918517+0000","last_unstale":"2024-04-18T23:11:54.672551+0000","last_undegraded":"2024-04-18T23:11:54.672551+0000","last_fullsized":"2024-04-18T23:11:54.672551+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.210659443,"stat_sum":{"num_bytes":6995968,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":16277,"num_write":54,"num_write_kb":8016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.d","version":"87'148","reported_seq":425,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-04-18T23:11:56.906516+0000","last_change":"2024-04-18T23:11:54.401188+0000","last_active":"2024-04-18T23:11:56.906516+0000","last_peered":"2024-04-18T23:11:56.906516+0000","last_clean":"2024-04-18T23:11:56.906516+0000","last_became_active":"2024-04-18T23:10:54.712663+0000","last_became_peered":"2024-04-18T23:10:54.712663+0000","last_unstale":"2024-04-18T23:11:56.906516+0000","last_undegraded":"2024-04-18T23:11:56.906516+0000","last_fullsized":"2024-04-18T23:11:56.906516+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:58.090670+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.620718035,"stat_sum":{"num_bytes":11730944,"num_objects":13,"num_object_clones":8,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":51,"num_read_kb":5297,"num_write":103,"num_write_kb":15088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.c","version":"87'61","reported_seq":328,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:54.678074+0000","last_change":"2024-04-18T23:11:54.389100+0000","last_active":"2024-04-18T23:11:54.678074+0000","last_peered":"2024-04-18T23:11:54.678074+0000","last_clean":"2024-04-18T23:11:54.678074+0000","last_became_active":"2024-04-18T23:10:54.717053+0000","last_became_peered":"2024-04-18T23:10:54.717053+0000","last_unstale":"2024-04-18T23:11:54.678074+0000","last_undegraded":"2024-04-18T23:11:54.678074+0000","last_fullsized":"2024-04-18T23:11:54.678074+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.60923718000000004,"stat_sum":{"num_bytes":6471680,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":1926,"num_write":35,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.b","version":"87'155","reported_seq":422,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:55.691790+0000","last_change":"2024-04-18T23:11:54.682530+0000","last_active":"2024-04-18T23:11:55.691790+0000","last_peered":"2024-04-18T23:11:55.691790+0000","last_clean":"2024-04-18T23:11:55.691790+0000","last_became_active":"2024-04-18T23:10:54.808152+0000","last_became_peered":"2024-04-18T23:10:54.808152+0000","last_unstale":"2024-04-18T23:11:55.691790+0000","last_undegraded":"2024-04-18T23:11:55.691790+0000","last_fullsized":"2024-04-18T23:11:55.691790+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:16.420923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.011405402,"stat_sum":{"num_bytes":13631488,"num_objects":11,"num_object_clones":6,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":33,"num_read_kb":8320,"num_write":65,"num_write_kb":13584,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.a","version":"87'24","reported_seq":304,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:55.160877+0000","last_change":"2024-04-18T23:11:53.982789+0000","last_active":"2024-04-18T23:11:55.160877+0000","last_peered":"2024-04-18T23:11:55.160877+0000","last_clean":"2024-04-18T23:11:55.160877+0000","last_became_active":"2024-04-18T23:10:54.764999+0000","last_became_peered":"2024-04-18T23:10:54.764999+0000","last_unstale":"2024-04-18T23:11:55.160877+0000","last_undegraded":"2024-04-18T23:11:55.160877+0000","last_fullsized":"2024-04-18T23:11:55.160877+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.30778924499999999,"stat_sum":{"num_bytes":720896,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":3750,"num_write":6,"num_write_kb":624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.9","version":"87'124","reported_seq":442,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:56.284800+0000","last_change":"2024-04-18T23:11:54.390988+0000","last_active":"2024-04-18T23:11:56.284800+0000","last_peered":"2024-04-18T23:11:56.284800+0000","last_clean":"2024-04-18T23:11:56.284800+0000","last_became_active":"2024-04-18T23:11:39.726824+0000","last_became_peered":"2024-04-18T23:11:39.726824+0000","last_unstale":"2024-04-18T23:11:56.284800+0000","last_undegraded":"2024-04-18T23:11:56.284800+0000","last_fullsized":"2024-04-18T23:11:56.284800+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:17.448637+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.72117339000000003,"stat_sum":{"num_bytes":5324800,"num_objects":13,"num_object_clones":7,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":55,"num_read_kb":10769,"num_write":81,"num_write_kb":7696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2867200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.8","version":"87'61","reported_seq":356,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:54.774100+0000","last_change":"2024-04-18T23:11:54.092481+0000","last_active":"2024-04-18T23:11:54.774100+0000","last_peered":"2024-04-18T23:11:54.774100+0000","last_clean":"2024-04-18T23:11:54.774100+0000","last_became_active":"2024-04-18T23:11:37.876250+0000","last_became_peered":"2024-04-18T23:11:37.876250+0000","last_unstale":"2024-04-18T23:11:54.774100+0000","last_undegraded":"2024-04-18T23:11:54.774100+0000","last_fullsized":"2024-04-18T23:11:54.774100+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.972562+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.31606408000000003,"stat_sum":{"num_bytes":1949696,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":21,"num_read_kb":2250,"num_write":54,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"1.0","version":"33'157","reported_seq":344,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:54.787799+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:11:54.787799+0000","last_peered":"2024-04-18T23:11:54.787799+0000","last_clean":"2024-04-18T23:11:54.787799+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:11:54.787799+0000","last_undegraded":"2024-04-18T23:11:54.787799+0000","last_fullsized":"2024-04-18T23:11:54.787799+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":147,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:54.673614+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:11:54.673614+0000","last_peered":"2024-04-18T23:11:54.673614+0000","last_clean":"2024-04-18T23:11:54.673614+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:11:54.673614+0000","last_undegraded":"2024-04-18T23:11:54.673614+0000","last_fullsized":"2024-04-18T23:11:54.673614+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"87'77","reported_seq":371,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:54.687813+0000","last_change":"2024-04-18T23:11:54.296088+0000","last_active":"2024-04-18T23:11:54.687813+0000","last_peered":"2024-04-18T23:11:54.687813+0000","last_clean":"2024-04-18T23:11:54.687813+0000","last_became_active":"2024-04-18T23:11:33.818951+0000","last_became_peered":"2024-04-18T23:11:33.818951+0000","last_unstale":"2024-04-18T23:11:54.687813+0000","last_undegraded":"2024-04-18T23:11:54.687813+0000","last_fullsized":"2024-04-18T23:11:54.687813+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:18.622494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.52634094899999995,"stat_sum":{"num_bytes":9797632,"num_objects":7,"num_object_clones":3,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":6012,"num_write":36,"num_write_kb":9872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:55.160538+0000","last_change":"2024-04-18T23:10:45.497972+0000","last_active":"2024-04-18T23:11:55.160538+0000","last_peered":"2024-04-18T23:11:55.160538+0000","last_clean":"2024-04-18T23:11:55.160538+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:11:55.160538+0000","last_undegraded":"2024-04-18T23:11:55.160538+0000","last_fullsized":"2024-04-18T23:11:55.160538+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:57.642474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"88'10","reported_seq":281,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:55.691588+0000","last_change":"2024-04-18T23:11:54.789266+0000","last_active":"2024-04-18T23:11:55.691588+0000","last_peered":"2024-04-18T23:11:55.691588+0000","last_clean":"2024-04-18T23:11:55.691588+0000","last_became_active":"2024-04-18T23:10:54.799189+0000","last_became_peered":"2024-04-18T23:10:54.799189+0000","last_unstale":"2024-04-18T23:11:55.691588+0000","last_undegraded":"2024-04-18T23:11:55.691588+0000","last_fullsized":"2024-04-18T23:11:55.691588+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.733712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.118318573,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":134,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:54.687728+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:11:54.687728+0000","last_peered":"2024-04-18T23:11:54.687728+0000","last_clean":"2024-04-18T23:11:54.687728+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:11:54.687728+0000","last_undegraded":"2024-04-18T23:11:54.687728+0000","last_fullsized":"2024-04-18T23:11:54.687728+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:14.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"87'44","reported_seq":347,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:56.284925+0000","last_change":"2024-04-18T23:11:53.976323+0000","last_active":"2024-04-18T23:11:56.284925+0000","last_peered":"2024-04-18T23:11:56.284925+0000","last_clean":"2024-04-18T23:11:56.284925+0000","last_became_active":"2024-04-18T23:11:38.818022+0000","last_became_peered":"2024-04-18T23:11:38.818022+0000","last_unstale":"2024-04-18T23:11:56.284925+0000","last_undegraded":"2024-04-18T23:11:56.284925+0000","last_fullsized":"2024-04-18T23:11:56.284925+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:18.578716+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.30645110800000003,"stat_sum":{"num_bytes":6438912,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":2792,"num_write":20,"num_write_kb":6288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":7716864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":133,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:55.260945+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:11:55.260945+0000","last_peered":"2024-04-18T23:11:55.260945+0000","last_clean":"2024-04-18T23:11:55.260945+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:11:55.260945+0000","last_undegraded":"2024-04-18T23:11:55.260945+0000","last_fullsized":"2024-04-18T23:11:55.260945+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.994274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":280,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-04-18T23:11:54.687997+0000","last_change":"2024-04-18T23:11:53.771290+0000","last_active":"2024-04-18T23:11:54.687997+0000","last_peered":"2024-04-18T23:11:54.687997+0000","last_clean":"2024-04-18T23:11:54.687997+0000","last_became_active":"2024-04-18T23:11:33.713370+0000","last_became_peered":"2024-04-18T23:11:33.713370+0000","last_unstale":"2024-04-18T23:11:54.687997+0000","last_undegraded":"2024-04-18T23:11:54.687997+0000","last_fullsized":"2024-04-18T23:11:54.687997+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:11:58.252919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014547539999999999,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"3.5","version":"88'117","reported_seq":409,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:56.226470+0000","last_change":"2024-04-18T23:11:54.394665+0000","last_active":"2024-04-18T23:11:56.226470+0000","last_peered":"2024-04-18T23:11:56.226470+0000","last_clean":"2024-04-18T23:11:56.226470+0000","last_became_active":"2024-04-18T23:11:39.923911+0000","last_became_peered":"2024-04-18T23:11:39.923911+0000","last_unstale":"2024-04-18T23:11:56.226470+0000","last_undegraded":"2024-04-18T23:11:56.226470+0000","last_fullsized":"2024-04-18T23:11:56.226470+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:22.435908+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.720096136,"stat_sum":{"num_bytes":6455296,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":40,"num_read_kb":12302,"num_write":57,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5685248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":144,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:55.691303+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:11:55.691303+0000","last_peered":"2024-04-18T23:11:55.691303+0000","last_clean":"2024-04-18T23:11:55.691303+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:11:55.691303+0000","last_undegraded":"2024-04-18T23:11:55.691303+0000","last_fullsized":"2024-04-18T23:11:55.691303+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"88'105","reported_seq":365,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:55.912421+0000","last_change":"2024-04-18T23:11:54.579717+0000","last_active":"2024-04-18T23:11:55.912421+0000","last_peered":"2024-04-18T23:11:55.912421+0000","last_clean":"2024-04-18T23:11:55.912421+0000","last_became_active":"2024-04-18T23:10:54.793319+0000","last_became_peered":"2024-04-18T23:10:54.793319+0000","last_unstale":"2024-04-18T23:11:55.912421+0000","last_undegraded":"2024-04-18T23:11:55.912421+0000","last_fullsized":"2024-04-18T23:11:55.912421+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:00.286059+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.81267847999999998,"stat_sum":{"num_bytes":9486336,"num_objects":9,"num_object_clones":5,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":16,"num_read_kb":7,"num_write":29,"num_write_kb":9968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":136,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:56.018531+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:11:56.018531+0000","last_peered":"2024-04-18T23:11:56.018531+0000","last_clean":"2024-04-18T23:11:56.018531+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:11:56.018531+0000","last_undegraded":"2024-04-18T23:11:56.018531+0000","last_fullsized":"2024-04-18T23:11:56.018531+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.503236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"88'206","reported_seq":470,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:55.791761+0000","last_change":"2024-04-18T23:11:55.083020+0000","last_active":"2024-04-18T23:11:55.791761+0000","last_peered":"2024-04-18T23:11:55.791761+0000","last_clean":"2024-04-18T23:11:55.791761+0000","last_became_active":"2024-04-18T23:10:54.693609+0000","last_became_peered":"2024-04-18T23:10:54.693609+0000","last_unstale":"2024-04-18T23:11:55.791761+0000","last_undegraded":"2024-04-18T23:11:55.791761+0000","last_fullsized":"2024-04-18T23:11:55.791761+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:01.820526+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.40953228000000003,"stat_sum":{"num_bytes":12664832,"num_objects":14,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":48,"num_read_kb":11703,"num_write":163,"num_write_kb":14880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":135,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:55.912345+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:11:55.912345+0000","last_peered":"2024-04-18T23:11:55.912345+0000","last_clean":"2024-04-18T23:11:55.912345+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:11:55.912345+0000","last_undegraded":"2024-04-18T23:11:55.912345+0000","last_fullsized":"2024-04-18T23:11:55.912345+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.869947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"87'42","reported_seq":346,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:56.197404+0000","last_change":"2024-04-18T23:11:54.088435+0000","last_active":"2024-04-18T23:11:56.197404+0000","last_peered":"2024-04-18T23:11:56.197404+0000","last_clean":"2024-04-18T23:11:56.197404+0000","last_became_active":"2024-04-18T23:11:39.717443+0000","last_became_peered":"2024-04-18T23:11:39.717443+0000","last_unstale":"2024-04-18T23:11:56.197404+0000","last_undegraded":"2024-04-18T23:11:56.197404+0000","last_fullsized":"2024-04-18T23:11:56.197404+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:02.399585+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.416605591,"stat_sum":{"num_bytes":3358720,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":1879,"num_write":18,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"c","length":"2"},{"start":"f","length":"1"},{"start":"11","length":"1"},{"start":"14","length":"1"},{"start":"17","length":"2"},{"start":"1a","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":62,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-18T23:11:56.326726+0000","last_change":"2024-04-18T23:11:33.519671+0000","last_active":"2024-04-18T23:11:56.326726+0000","last_peered":"2024-04-18T23:11:56.326726+0000","last_clean":"2024-04-18T23:11:56.326726+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:11:56.326726+0000","last_undegraded":"2024-04-18T23:11:56.326726+0000","last_fullsized":"2024-04-18T23:11:56.326726+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":96993280,"num_objects":103,"num_object_clones":53,"num_object_copies":721,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":5,"num_read":400,"num_read_kb":96730,"num_write":731,"num_write_kb":114912,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":169684992,"data_stored":228286464,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0},"log_size":1277,"ondisk_log_size":1277,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808838,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38048,"kb_used_data":9492,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94333792,"statfs":{"total":96636764160,"available":96597803008,"internally_reserved":0,"allocated":9719808,"data_stored":12431693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280531,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41584,"kb_used_data":10408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330256,"statfs":{"total":96636764160,"available":96594182144,"internally_reserved":0,"allocated":10657792,"data_stored":12179708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280531,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40540,"kb_used_data":9356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331300,"statfs":{"total":96636764160,"available":96595251200,"internally_reserved":0,"allocated":9580544,"data_stored":10963740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313234,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50264,"kb_used_data":19076,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94321576,"statfs":{"total":96636764160,"available":96585293824,"internally_reserved":0,"allocated":19533824,"data_stored":24000764,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313234,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46504,"kb_used_data":15320,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325336,"statfs":{"total":96636764160,"available":96589144064,"internally_reserved":0,"allocated":15687680,"data_stored":20531452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313235,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34988,"kb_used_data":7964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336852,"statfs":{"total":96636764160,"available":96600936448,"internally_reserved":0,"allocated":8155136,"data_stored":9234684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313235,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47212,"kb_used_data":16024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324628,"statfs":{"total":96636764160,"available":96588419072,"internally_reserved":0,"allocated":16408576,"data_stored":20652365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,1,0,9],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313235,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35008,"kb_used_data":7980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336832,"statfs":{"total":96636764160,"available":96600915968,"internally_reserved":0,"allocated":8171520,"data_stored":9976060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313233,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47344,"kb_used_data":16160,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324496,"statfs":{"total":96636764160,"available":96588283904,"internally_reserved":0,"allocated":16547840,"data_stored":21260540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313233,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37684,"kb_used_data":10660,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334156,"statfs":{"total":96636764160,"available":96598175744,"internally_reserved":0,"allocated":10915840,"data_stored":14084348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313234,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39876,"kb_used_data":8688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331964,"statfs":{"total":96636764160,"available":96595931136,"internally_reserved":0,"allocated":8896512,"data_stored":12208380,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,1],"upper_bound":64},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280531,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41900,"kb_used_data":10716,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329940,"statfs":{"total":96636764160,"available":96593858560,"internally_reserved":0,"allocated":10973184,"data_stored":13896476,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280531,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48328,"kb_used_data":17148,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323512,"statfs":{"total":96636764160,"available":96587276288,"internally_reserved":0,"allocated":17559552,"data_stored":24375629,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280532,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41212,"kb_used_data":10028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330628,"statfs":{"total":96636764160,"available":96594563072,"internally_reserved":0,"allocated":10268672,"data_stored":12855710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280531,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41540,"kb_used_data":10364,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330300,"statfs":{"total":96636764160,"available":96594227200,"internally_reserved":0,"allocated":10612736,"data_stored":13871356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280531,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38692,"kb_used_data":7508,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94333148,"statfs":{"total":96636764160,"available":96597143552,"internally_reserved":0,"allocated":7688192,"data_stored":8792478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":18251776,"data_stored":23257088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":14405632,"data_stored":19787776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9158656,"data_stored":12623872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6873088,"data_stored":8491008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":15110144,"data_stored":19898368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6889472,"data_stored":9232384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16261120,"data_stored":23621632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8953856,"data_stored":12091392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":15265792,"data_stored":20516864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":9330688,"data_stored":13127680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":6373376,"data_stored":8028160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":9633792,"data_stored":13340672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":7614464,"data_stored":11464704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":7766016,"data_stored":9691136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":9375744,"data_stored":11436032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":8421376,"data_stored":11677696,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0}]}} 2024-04-18T23:11:58.970 INFO:tasks.ceph.ceph_manager.ceph:would drop below min_pgs, proj 6, currently 16 2024-04-18T23:11:59.031 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:59.028+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:59.041 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:59.041 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:59.050 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:11:59.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:11:59.132 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:11:59.132+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:11:59.180 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:59.180 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:59.193 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:11:59.201 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:11:59.201 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:11:59.221 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:11:59.222 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:11:59.232 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:11:59.232+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:11:59.244 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:11:59.244 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:11:59.263 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:11:59.263 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:11:59.281 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:11:59.282 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:11:59.303 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:11:59.304 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:11:59.316 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:11:59.316 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:11:59.325 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:11:59.325 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:11:59.330 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:11:59.332 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:11:59.331+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:11:59.345 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:11:59.345 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:11:59.365 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:11:59.365 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:11:59.386 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:11:59.386 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:11:59.408 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:11:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:11:59.429 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:11:59.429 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:11:59.433 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:59.434+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:59.448 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:11:59.448 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:202 3.ds0 3:ba4438ef:::smithi14230322-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 58.809531030999999, 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.726850518, 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.459 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 202 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.460 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647077+0000", 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.497e-06 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647086+0000", 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5490000000000006e-06 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647105+0000", 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8933e-05 2024-04-18T23:11:59.461 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647120+0000", 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4942e-05 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752033+0000", 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10491311 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752334+0000", 2024-04-18T23:11:59.462 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030070900000000002 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.072768+0000", 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32043411399999999 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.075547+0000", 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027797260000000002 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.463 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.373671+0000", 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29812367699999998 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.373926+0000", 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025526100000000001 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:203 3.ds0 3:ba4438ef:::smithi14230322-48:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647172+0000", 2024-04-18T23:11:59.464 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 58.809435078999996, 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.42781979999999997, 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 203 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647172+0000", 2024-04-18T23:11:59.465 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647172+0000", 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647174+0000", 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8369999999999999e-06 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647180+0000", 2024-04-18T23:11:59.466 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9529999999999996e-06 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647201+0000", 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1501999999999998e-05 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647218+0000", 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.647e-05 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.467 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.753422+0000", 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10620457699999999 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.753567+0000", 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014483399999999999 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.074716+0000", 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.321149236 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.468 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.074851+0000", 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000135258 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.074992+0000", 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014013299999999999 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:319 3.ds0 3:ba4438ef:::smithi14230322-48:7 [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609786+0000", 2024-04-18T23:11:59.469 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.846820577000003, 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50925149999999997, 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 319 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609786+0000", 2024-04-18T23:11:59.470 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609786+0000", 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609788+0000", 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3999999999999999e-06 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.471 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609795+0000", 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.3200000000000002e-06 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609813+0000", 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8493000000000001e-05 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609830+0000", 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6062e-05 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.472 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.610105+0000", 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000275467 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.610126+0000", 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1262000000000001e-05 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.814838+0000", 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.204711587 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.473 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.815034+0000", 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000196579 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119038+0000", 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30400333000000002 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.474 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.475 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.475 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:323 3.ds0 3:b8bdcfdb:::smithi14230322-44:head [delete] snapc 8=[8,7,4] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:11:59.475 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:11:59.475 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 48.846745925999997, 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81471389100000002, 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 323 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.476 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609863+0000", 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6360000000000001e-06 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609869+0000", 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4699999999999999e-06 2024-04-18T23:11:59.477 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609890+0000", 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0496000000000002e-05 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609905+0000", 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5539000000000001e-05 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.614202+0000", 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042970489999999998 2024-04-18T23:11:59.478 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.614219+0000", 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7373000000000001e-05 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119137+0000", 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50491712899999996 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119293+0000", 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015688099999999999 2024-04-18T23:11:59.479 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120709+0000", 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014158669999999999 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.122707+0000", 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019980140000000002 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424382+0000", 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30167486399999999 2024-04-18T23:11:59.480 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424575+0000", 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000192573 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:389 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~458752] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.672684+0000", 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.783922973999999, 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71125476700000001, 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.481 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 389 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.672684+0000", 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.482 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.672684+0000", 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.672687+0000", 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.439e-06 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.673226+0000", 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00053900300000000003 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.673261+0000", 2024-04-18T23:11:59.483 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5169000000000002e-05 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.673279+0000", 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7353000000000001e-05 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.676118+0000", 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002839453 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.676134+0000", 2024-04-18T23:11:59.484 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5489999999999999e-05 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.874209+0000", 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.198074958 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.874392+0000", 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018306999999999999 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.876303+0000", 2024-04-18T23:11:59.485 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001910878 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.881915+0000", 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0056125589999999996 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.383786+0000", 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50187064000000003 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.486 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.383939+0000", 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000152755 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:390 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~475136] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.675363+0000", 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.781243750000002, 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.80904401299999995, 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.487 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 390 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675363+0000", 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675363+0000", 2024-04-18T23:11:59.488 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675365+0000", 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1569999999999998e-06 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675936+0000", 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00057051499999999995 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.489 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675968+0000", 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1779000000000002e-05 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675983+0000", 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5313000000000001e-05 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.873642+0000", 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.197659001 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:59.490 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.873655+0000", 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3073e-05 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.877399+0000", 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0037437019999999998 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.877612+0000", 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021331900000000001 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.491 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.879219+0000", 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001607266 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.881917+0000", 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026975240000000002 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484280+0000", 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60236371499999997 2024-04-18T23:11:59.492 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484407+0000", 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000126649 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:391 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~294912] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.677038+0000", 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.779568961999999, 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.80775349699999999, 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.493 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 391 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677038+0000", 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.494 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677038+0000", 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677040+0000", 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0059999999999999e-06 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677406+0000", 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00036561500000000002 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.495 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677440+0000", 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4972999999999999e-05 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777555+0000", 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100114034 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.880210+0000", 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10265596 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.496 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.880389+0000", 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017841299999999999 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.881177+0000", 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00078838699999999999 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.183072+0000", 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30189508599999998 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.497 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484661+0000", 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30158817300000001 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484791+0000", 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013085000000000001 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:392 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [setxattr _header (57) in=7b] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:59.498 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.777603+0000", 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.679004159000002, 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.70764648600000002, 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 392 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.499 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777603+0000", 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777603+0000", 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777605+0000", 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0439999999999998e-06 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.500 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777618+0000", 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3169e-05 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777650+0000", 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2492e-05 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777674+0000", 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3896e-05 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.501 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980335+0000", 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.202661059 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.980638+0000", 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030296400000000002 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.981741+0000", 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011024609999999999 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.502 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.183074+0000", 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20133356599999999 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.485059+0000", 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30198478499999998 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.485249+0000", 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019005000000000001 2024-04-18T23:11:59.503 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:393 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.777705+0000", 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.678902115, 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0113445000000001, 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.504 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 393 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777705+0000", 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777705+0000", 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.505 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777706+0000", 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.626e-06 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777716+0000", 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0146000000000001e-05 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777742+0000", 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5236e-05 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.506 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777762+0000", 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9879999999999999e-05 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.982379+0000", 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20461718700000001 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.982705+0000", 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032662899999999999 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.507 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.486129+0000", 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50342405499999998 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.486297+0000", 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016788799999999999 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.789049+0000", 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30275185300000002 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.508 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:426 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [copy-from ver 16] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:21.537160+0000", 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 37.919446393999998, 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.52374796700000004, 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 426 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.509 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537160+0000", 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537160+0000", 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537163+0000", 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3750000000000001e-06 2024-04-18T23:11:59.510 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537176+0000", 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2755000000000001e-05 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537201+0000", 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5451000000000001e-05 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537212+0000", 2024-04-18T23:11:59.511 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0614e-05 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537295+0000", 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3707000000000003e-05 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537302+0000", 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.0729999999999999e-06 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.741218+0000", 2024-04-18T23:11:59.512 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20391555 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.741402+0000", 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018423199999999999 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.953934+0000", 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21253224000000001 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.513 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.058114+0000", 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104179409 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.060716+0000", 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026026000000000001 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.060908+0000", 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019196099999999999 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.514 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:427 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:21.537234+0000", 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 37.919373073000003, 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.52066839700000001, 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 427 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.515 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537234+0000", 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537234+0000", 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537234+0000", 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2300000000000002e-07 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.516 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537237+0000", 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4140000000000001e-06 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537242+0000", 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.6500000000000001e-06 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537246+0000", 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6839999999999998e-06 2024-04-18T23:11:59.517 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537413+0000", 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016680199999999999 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.537424+0000", 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1379999999999999e-05 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.741750+0000", 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20432588900000001 2024-04-18T23:11:59.518 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:21.741914+0000", 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016375299999999999 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.057567+0000", 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31565273700000002 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.057722+0000", 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000155883 2024-04-18T23:11:59.519 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:22.057902+0000", 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017978199999999999 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:502 3.ds0 3:ba4438ef:::smithi14230322-48:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.529587+0000", 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.927019600000001, 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.58018316299999995, 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.520 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 502 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529587+0000", 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.521 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529587+0000", 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529588+0000", 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.009e-06 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529594+0000", 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.254e-06 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.522 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529607+0000", 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3730999999999999e-05 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529616+0000", 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.8570000000000001e-06 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.531417+0000", 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018004500000000001 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.523 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.531690+0000", 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00027340099999999999 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.109770+0000", 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.57808046099999999 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:503 3.ds0 3:ba4438ef:::smithi14230322-48:head [read 0~32768,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:59.524 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.529671+0000", 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.926936308999998, 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.58061668499999997, 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 503 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529671+0000", 2024-04-18T23:11:59.525 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529671+0000", 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529671+0000", 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.4900000000000005e-07 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.526 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529676+0000", 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6779999999999999e-06 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.529687+0000", 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1404e-05 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629836+0000", 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100148557 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.527 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.633504+0000", 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0036684729999999998 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.633773+0000", 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026893700000000002 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.110287+0000", 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.47651378700000002 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.528 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:509 3.ds0 3:bb40a4d8:::smithi14230322-41:head [copy-from ver 20] snapc 15=[15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.629899+0000", 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.826708025999999, 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.62332154699999998, 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 509 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.529 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629899+0000", 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629899+0000", 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629902+0000", 2024-04-18T23:11:59.530 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7329999999999999e-06 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629919+0000", 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.734e-05 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629948+0000", 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8782999999999999e-05 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.531 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629961+0000", 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3349999999999999e-05 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.735207+0000", 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105245482 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.735530+0000", 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032370599999999999 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.532 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.116389+0000", 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.38085889099999998 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.120910+0000", 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0045208920000000003 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.252995+0000", 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.13208513499999999 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.533 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.253220+0000", 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022523500000000001 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:510 3.ds0 3:bb40a4d8:::smithi14230322-41:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:31.629979+0000", 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 27.826628297999999, 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49023087100000001, 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.534 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 510 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629979+0000", 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629979+0000", 2024-04-18T23:11:59.535 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629980+0000", 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.06e-06 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629984+0000", 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6430000000000003e-06 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.629995+0000", 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1060000000000001e-05 2024-04-18T23:11:59.536 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.730164+0000", 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100168985 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.736017+0000", 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0058526969999999996 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:31.736111+0000", 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.3894000000000006e-05 2024-04-18T23:11:59.537 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.119824+0000", 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.383712834 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.120018+0000", 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000194465 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.538 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:32.120209+0000", 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019123299999999999 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:703 3.ds0 3:b210783d:::smithi14230322-30:head [append 0~737280] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.852388+0000", 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.6042189489999998, 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71389459600000005, 2024-04-18T23:11:59.539 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 703 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.852388+0000", 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.540 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.852388+0000", 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.852391+0000", 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8399999999999999e-06 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853230+0000", 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00083946900000000002 2024-04-18T23:11:59.541 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853263+0000", 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2502999999999997e-05 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853278+0000", 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5619000000000001e-05 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853408+0000", 2024-04-18T23:11:59.542 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000129617 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853578+0000", 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000170366 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.855600+0000", 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020215070000000001 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.858840+0000", 2024-04-18T23:11:59.543 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00324041 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566135+0000", 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.70729440899999996 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566282+0000", 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000147856 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.544 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:704 3.ds0 3:b210783d:::smithi14230322-30:head [append 0~557056] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.858086+0000", 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.5985212759999996, 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.70870384399999997, 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 704 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.545 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.858086+0000", 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.858086+0000", 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.858088+0000", 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.74e-06 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.546 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.858981+0000", 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00089311399999999997 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.859008+0000", 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.654e-05 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.859027+0000", 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9446e-05 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.547 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.859142+0000", 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0001141 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.859325+0000", 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018314899999999999 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.861029+0000", 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0017047080000000001 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.548 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.862966+0000", 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019364689999999999 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566584+0000", 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.70361773000000005 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566789+0000", 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020584800000000001 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.549 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:705 3.ds0 3:b210783d:::smithi14230322-30:head [append 0~606208] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.960181+0000", 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.4964253430000003, 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60755138900000005, 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 705 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.550 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.960181+0000", 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.960181+0000", 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.960188+0000", 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.477e-06 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.551 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961026+0000", 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00083817899999999999 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961058+0000", 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1622000000000003e-05 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961080+0000", 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2237000000000001e-05 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.552 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.059700+0000", 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.098619763999999999 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.059904+0000", 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000203912 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.061509+0000", 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001605572 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.553 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.064213+0000", 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027033030000000002 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.567498+0000", 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50328580199999995 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.567733+0000", 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00023452100000000001 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.554 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:706 3.ds0 3:b210783d:::smithi14230322-30:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.961124+0000", 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.4954827030000004, 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60700343700000003, 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 706 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.555 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961124+0000", 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961124+0000", 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961126+0000", 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0499999999999999e-06 2024-04-18T23:11:59.556 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961135+0000", 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9760000000000001e-06 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961155+0000", 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9470000000000002e-05 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961172+0000", 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6976000000000001e-05 2024-04-18T23:11:59.557 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.062828+0000", 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10165663 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.063043+0000", 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021471500000000001 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.063779+0000", 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000736475 2024-04-18T23:11:59.558 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.164642+0000", 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100862942 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.567987+0000", 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40334487899999999 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.568128+0000", 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014032399999999999 2024-04-18T23:11:59.559 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:707 3.ds0 3:b210783d:::smithi14230322-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.961197+0000", 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.4954103280000002, 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81019067, 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.560 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 707 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961197+0000", 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961197+0000", 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961198+0000", 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5200000000000001e-06 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961208+0000", 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.022e-05 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961228+0000", 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0078999999999999e-05 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961244+0000", 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5566e-05 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.164394+0000", 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20314963 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.164576+0000", 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018204500000000001 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.568877+0000", 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.404301311 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.569042+0000", 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016466199999999999 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.771387+0000", 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20234563699999999 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.564 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.565 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.565 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:59.566 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:11:59.566 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:11:59.567 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:11:59.567 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:11:59.567 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:11:59.568 INFO:tasks.rados.rados.0.smithi142.stdout:327: done (3 left) 2024-04-18T23:11:59.568 INFO:tasks.rados.rados.0.smithi142.stdout:328: done (2 left) 2024-04-18T23:11:59.568 INFO:tasks.rados.rados.0.smithi142.stdout:330: done (1 left) 2024-04-18T23:11:59.568 INFO:tasks.rados.rados.0.smithi142.stdout:331: done (0 left) 2024-04-18T23:11:59.568 INFO:tasks.rados.rados.0.smithi142.stdout:332: read oid 23 snap -1 2024-04-18T23:11:59.568 INFO:tasks.rados.rados.0.smithi142.stdout:332: expect (ObjNum 123 snap 31 seq_num 123) 2024-04-18T23:11:59.568 INFO:tasks.rados.rados.0.smithi142.stdout:333: snap_remove snap 29 2024-04-18T23:11:59.569 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:11:59.569 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:11:59.570 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:59.562+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:59.667 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:11:59.662+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:11:59.695 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:59.696 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:59.696 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:11:59.696 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:59.710 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:11:59.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T23:11:59.768 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:11:59.766+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:11:59.827 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:59.827 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:11:59.827 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:11:59.827 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:11:59.827 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:11:59.841 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:11:59.869 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:11:59.866+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:11:59.969 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:11:59.969 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:11:59.969 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:192 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 59.408593590999999, 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30501581700000002, 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.970 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 192 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:11:59.971 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560439+0000", 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6029999999999999e-06 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560450+0000", 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0835999999999999e-05 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560478+0000", 2024-04-18T23:11:59.972 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7619000000000001e-05 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560489+0000", 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0998999999999999e-05 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560680+0000", 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019082800000000001 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560688+0000", 2024-04-18T23:11:59.973 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2260000000000005e-06 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662105+0000", 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10141672 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662220+0000", 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115872 2024-04-18T23:11:59.974 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.865454+0000", 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20323311399999999 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:193 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:59.975 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 59.408484846, 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30581751899999998, 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 193 2024-04-18T23:11:59.976 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560548+0000", 2024-04-18T23:11:59.977 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560554+0000", 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2230000000000004e-06 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560566+0000", 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2622999999999999e-05 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.978 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560575+0000", 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1589999999999993e-06 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563654+0000", 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030794709999999999 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563665+0000", 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0708e-05 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.979 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662682+0000", 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099017665000000005 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.663181+0000", 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049841999999999996 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.866364+0000", 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20318328699999999 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.980 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:210 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 59.408387238000003, 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.41278097800000002, 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 210 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.981 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:11:59.982 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560645+0000", 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1829999999999999e-06 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560652+0000", 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3500000000000002e-06 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.983 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560669+0000", 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6833e-05 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560682+0000", 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3585e-05 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560856+0000", 2024-04-18T23:11:59.984 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017400200000000001 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560874+0000", 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8386e-05 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767285+0000", 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20641079700000001 2024-04-18T23:11:59.985 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767423+0000", 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013740399999999999 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.973425+0000", 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20600243800000001 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.986 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:211 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 59.408298942000002, 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51049911299999995, 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 211 2024-04-18T23:11:59.987 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560734+0000", 2024-04-18T23:11:59.988 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-06 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560741+0000", 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1559999999999998e-06 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560758+0000", 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7382999999999999e-05 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.989 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560775+0000", 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7051999999999999e-05 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560940+0000", 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164659 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560949+0000", 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7630000000000006e-06 2024-04-18T23:11:59.990 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767943+0000", 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.206994276 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.768107+0000", 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164358 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.071232+0000", 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30312427600000003 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.991 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:359 3.cs0 3:35606aaf:::smithi14230322-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 46.772782992000003, 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.36957785599999998, 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.992 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 359 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.993 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196249+0000", 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5700000000000002e-07 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196253+0000", 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8979999999999998e-06 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196271+0000", 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8270999999999998e-05 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.994 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196280+0000", 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.85e-06 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196650+0000", 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00037011599999999998 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196698+0000", 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8060999999999999e-05 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.995 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565575+0000", 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.36887705999999998 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565695+0000", 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119663 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565826+0000", 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013118 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:11:59.996 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:11:59.997 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.997 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.997 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:384 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~442368] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:11:59.997 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:11:59.997 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.304489687999997, 2024-04-18T23:11:59.997 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51544569399999995, 2024-04-18T23:11:59.997 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:11:59.997 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:11:59.997 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 384 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.998 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664544+0000", 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6240000000000002e-06 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665037+0000", 2024-04-18T23:11:59.999 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049266399999999997 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665070+0000", 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2963000000000003e-05 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665089+0000", 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8672e-05 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.000 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665233+0000", 2024-04-18T23:12:00.001 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014389900000000001 2024-04-18T23:12:00.001 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.001 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.001 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:12:00.001 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665245+0000", 2024-04-18T23:12:00.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2446000000000001e-05 2024-04-18T23:12:00.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.002 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.002 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769742+0000", 2024-04-18T23:12:00.002 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104497225 2024-04-18T23:12:00.002 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.002 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.770038+0000", 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029553099999999997 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.772939+0000", 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029013009999999998 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.003 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781282+0000", 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0083427859999999996 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.179880+0000", 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.398598591 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.179987+0000", 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010699199999999999 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.004 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:385 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~720896] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.300416063, 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61243552499999998, 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 385 2024-04-18T23:12:00.005 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:12:00.006 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668618+0000", 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5129999999999999e-06 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669576+0000", 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000957726 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.007 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.008 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669605+0000", 2024-04-18T23:12:00.008 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9566000000000001e-05 2024-04-18T23:12:00.008 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.008 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.008 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.009 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669622+0000", 2024-04-18T23:12:00.009 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6409000000000002e-05 2024-04-18T23:12:00.009 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.009 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.009 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.010 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.775317+0000", 2024-04-18T23:12:00.010 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105695331 2024-04-18T23:12:00.010 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.010 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.775501+0000", 2024-04-18T23:12:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018403000000000001 2024-04-18T23:12:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777261+0000", 2024-04-18T23:12:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001759652 2024-04-18T23:12:00.011 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781284+0000", 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040233049999999996 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.280844+0000", 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49956001900000002 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281051+0000", 2024-04-18T23:12:00.012 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020697399999999999 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:386 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~49152] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.299378992999998, 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61210873300000002, 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.013 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 386 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.014 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669654+0000", 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.297e-06 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669730+0000", 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6474e-05 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669748+0000", 2024-04-18T23:12:00.015 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7598999999999999e-05 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769883+0000", 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100135671 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.778786+0000", 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0089029639999999993 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.779085+0000", 2024-04-18T23:12:00.016 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029830699999999999 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.780258+0000", 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011728070000000001 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783446+0000", 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031888179999999999 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.017 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281593+0000", 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49814646800000001 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281761+0000", 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016832800000000001 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:387 3.fs0 3:f47c3018:::smithi14230322-43:head [setxattr _header (57) in=7b] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.199108437, 2024-04-18T23:12:00.018 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51243170900000001, 2024-04-18T23:12:00.019 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.019 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.019 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.019 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.019 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.019 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.019 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 387 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.022 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769925+0000", 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7799999999999999e-06 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769934+0000", 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6460000000000005e-06 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.023 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769956+0000", 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1216000000000002e-05 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769967+0000", 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-05 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781530+0000", 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.01156212 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.024 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781688+0000", 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015813200000000001 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.782502+0000", 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00081454100000000001 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783689+0000", 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001186552 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.025 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.282098+0000", 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49840956600000003 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.282355+0000", 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025625600000000001 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:388 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:12:00.026 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 43.199051009000001, 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81817408400000002, 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 388 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.027 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769981+0000", 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6400000000000001e-07 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769985+0000", 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7689999999999998e-06 2024-04-18T23:12:00.028 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769996+0000", 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.075e-05 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.770003+0000", 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5769999999999996e-06 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783045+0000", 2024-04-18T23:12:00.029 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013041308999999999 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783303+0000", 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025812900000000001 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.283544+0000", 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50024145900000005 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.030 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.283825+0000", 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028086099999999998 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.588154+0000", 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30432946599999999 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.031 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.fs0 51/31 ECSubReadReply(tid=28, attrs_read=0))", 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.683939541999997, 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30201136499999998, 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.032 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.033 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285093+0000", 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.08e-06 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285111+0000", 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8272000000000001e-05 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285129+0000", 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7706999999999998e-05 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.034 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285140+0000", 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1472e-05 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587038+0000", 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30189753600000002 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587103+0000", 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5297999999999997e-05 2024-04-18T23:12:00.035 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpRead(3.fs0 51/31 ECSubRead(tid=28, to_read={3:f47c3018:::smithi14230322-43:head=0,4096,0}, subchunks={3:f47c3018:::smithi14230322-43:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 42.683339338000003, 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30165329400000002, 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.036 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.037 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285692+0000", 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285725+0000", 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.3365999999999999e-05 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.038 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.385871+0000", 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100145397 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587204+0000", 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.201332972 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587345+0000", 2024-04-18T23:12:00.039 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000141559 2024-04-18T23:12:00.040 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.040 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.040 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.040 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.040 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.040 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 61/31 ECSubWriteReply(tid=39, last_complete=61'15, committed=1, applied=1))", 2024-04-18T23:12:00.040 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:26.653658+0000", 2024-04-18T23:12:00.040 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 33.315373528000002, 2024-04-18T23:12:00.040 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50232716700000002, 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653658+0000", 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653658+0000", 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.041 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653660+0000", 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2170000000000001e-06 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653670+0000", 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0183000000000001e-05 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.042 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653689+0000", 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9026e-05 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653701+0000", 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1922999999999999e-05 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:27.154982+0000", 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50128102200000002 2024-04-18T23:12:00.043 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:27.155985+0000", 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001002796 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 61/31 ECSubWriteReply(tid=39, last_complete=61'15, committed=1, applied=1))", 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:26.653748+0000", 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 33.315283876999999, 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.301973926, 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.044 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653748+0000", 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653748+0000", 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653749+0000", 2024-04-18T23:12:00.045 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2270000000000001e-06 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653755+0000", 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3409999999999999e-06 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653771+0000", 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6300999999999999e-05 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.046 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653786+0000", 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4236999999999999e-05 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.955692+0000", 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30190629699999999 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.955721+0000", 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9522999999999998e-05 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.047 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:561 3.cs0 3:35606aaf:::smithi14230322-20:head [append 0~442368] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.524261+0000", 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.444770868999999, 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31889435799999999, 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 561 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.048 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524261+0000", 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524261+0000", 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.049 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524263+0000", 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8040000000000002e-06 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524799+0000", 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00053593200000000003 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.050 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524834+0000", 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4693000000000001e-05 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524852+0000", 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7612999999999999e-05 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.525052+0000", 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020047899999999999 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.051 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.525066+0000", 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3937000000000001e-05 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.629867+0000", 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10480112 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.630105+0000", 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00023814499999999999 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.052 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.631631+0000", 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015257669999999999 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.641942+0000", 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010310784 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.842982+0000", 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20104003100000001 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.053 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.843155+0000", 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000173053 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:562 3.cs0 3:35606aaf:::smithi14230322-20:head [append 0~540672] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.531229+0000", 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.437802549000001, 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61875124800000003, 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.054 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 562 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531229+0000", 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531229+0000", 2024-04-18T23:12:00.055 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531231+0000", 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0530000000000001e-06 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531984+0000", 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00075264199999999998 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.532012+0000", 2024-04-18T23:12:00.056 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8932999999999999e-05 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.532029+0000", 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6203000000000001e-05 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.638598+0000", 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.106569199 2024-04-18T23:12:00.057 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.638913+0000", 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031543000000000001 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.640689+0000", 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001775715 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.742333+0000", 2024-04-18T23:12:00.058 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101644006 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.149762+0000", 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40742915800000001 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.149980+0000", 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021790899999999999 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.059 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:563 3.cs0 3:35606aaf:::smithi14230322-20:head [append 0~622592] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.538968+0000", 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.430063315000002, 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.68520978200000004, 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 563 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.060 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.538968+0000", 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.538968+0000", 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.061 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.538970+0000", 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0559999999999999e-06 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539699+0000", 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00072919900000000004 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539729+0000", 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9899999999999998e-05 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.062 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539744+0000", 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4708999999999999e-05 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.943419+0000", 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.403674637 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.943605+0000", 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018626 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.063 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.945303+0000", 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001697893 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148746+0000", 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20344289300000001 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.224013+0000", 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.075267071000000005 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.064 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.224178+0000", 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016516400000000001 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:564 3.cs0 3:35606aaf:::smithi14230322-20:head [setxattr _header (57) in=7b] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.539769+0000", 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.429261995000001, 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.78455714300000001, 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.065 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 564 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539769+0000", 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539769+0000", 2024-04-18T23:12:00.066 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539771+0000", 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2470000000000001e-06 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539777+0000", 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0730000000000001e-06 2024-04-18T23:12:00.067 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539788+0000", 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0924000000000001e-05 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539795+0000", 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8159999999999994e-06 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.046902+0000", 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50710653999999999 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.068 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.047084+0000", 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000181609 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.047936+0000", 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00085221099999999996 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148748+0000", 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100812007 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.069 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.324156+0000", 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17540829599999999 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.324327+0000", 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017042000000000001 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.070 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:565 3.cs0 3:35606aaf:::smithi14230322-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.539807+0000", 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.429224409, 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0887034069999999, 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 565 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539807+0000", 2024-04-18T23:12:00.071 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539807+0000", 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539808+0000", 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9299999999999997e-07 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539811+0000", 2024-04-18T23:12:00.072 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5700000000000001e-06 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539822+0000", 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0467e-05 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539829+0000", 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5870000000000004e-06 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148486+0000", 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60865698899999998 2024-04-18T23:12:00.073 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148648+0000", 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016127099999999999 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.326655+0000", 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17800708000000001 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.326971+0000", 2024-04-18T23:12:00.074 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031639400000000001 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.628510+0000", 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30153925599999998 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 72/31 ECSubWrite(tid=224, reqid=osd.4.0:222, at_version=0'0, trim_to=0'0, roll_forward_to=72'111))", 2024-04-18T23:12:00.075 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:39.511725+0000", 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 20.457306761000002, 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30299252599999998, 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511725+0000", 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.076 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511725+0000", 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511726+0000", 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1999999999999998e-07 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511748+0000", 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2276000000000002e-05 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.077 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511772+0000", 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3932999999999999e-05 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.611922+0000", 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100150379 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.712249+0000", 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100326822 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.078 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.712286+0000", 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6962999999999998e-05 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.814658+0000", 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102371988 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.814717+0000", 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9244999999999999e-05 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.079 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.080 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.080 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:00.081 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:12:00.082 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:00.079+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:00.182 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:00.183+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:00.240 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:12:00.241 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:12:00.241 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:00.255 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-18T23:12:00.298 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:00.299+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:00.384 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:00.382+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:00.421 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:00.421 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:12:00.421 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:12:00.421 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:12:00.421 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:00.435 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-18T23:12:00.484 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:00.482+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:00.519 INFO:tasks.rados.rados.0.smithi142.stdout:333: done (1 left) 2024-04-18T23:12:00.519 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 45 current snap is 32 2024-04-18T23:12:00.519 INFO:tasks.rados.rados.0.smithi142.stdout:334: seq_num 124 ranges {2473984=540672,3014656=720896,3735552=786432,4521984=49152} 2024-04-18T23:12:00.520 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-04-18T23:12:00.542 INFO:tasks.rados.rados.0.smithi142.stdout:334: writing smithi14230322-45 from 2473984 to 3014656 tid 1 2024-04-18T23:12:00.548 INFO:tasks.rados.rados.0.smithi142.stdout:334: writing smithi14230322-45 from 3014656 to 3735552 tid 2 2024-04-18T23:12:00.554 INFO:tasks.rados.rados.0.smithi142.stdout:334: writing smithi14230322-45 from 3735552 to 4521984 tid 3 2024-04-18T23:12:00.555 INFO:tasks.rados.rados.0.smithi142.stdout:334: writing smithi14230322-45 from 4521984 to 4571136 tid 4 2024-04-18T23:12:00.555 INFO:tasks.rados.rados.0.smithi142.stdout:335: rollback oid 35 current snap is 32 2024-04-18T23:12:00.555 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 35 to 17 2024-04-18T23:12:00.555 INFO:tasks.rados.rados.0.smithi142.stdout:336: snap_create 2024-04-18T23:12:00.555 INFO:tasks.rados.rados.0.smithi142.stdout:332: expect (ObjNum 123 snap 31 seq_num 123) 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:636 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~114688,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:44.611449+0000", 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.944271941, 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.407104254, 2024-04-18T23:12:00.559 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 636 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611449+0000", 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.560 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611449+0000", 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611452+0000", 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4470000000000002e-06 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611468+0000", 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5716e-05 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.561 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611504+0000", 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5963000000000001e-05 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611522+0000", 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8275999999999999e-05 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611614+0000", 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1224999999999995e-05 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.562 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611622+0000", 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9859999999999994e-06 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.712617+0000", 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100995288 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.712794+0000", 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017712699999999999 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.563 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:45.018553+0000", 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.305759226 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:637 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~114688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:44.611560+0000", 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.944160435000001, 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40760569800000002, 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:00.564 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 637 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611560+0000", 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611560+0000", 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.565 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611561+0000", 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6600000000000002e-07 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611564+0000", 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6909999999999999e-06 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611569+0000", 2024-04-18T23:12:00.566 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6659999999999999e-06 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611572+0000", 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.174e-06 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611687+0000", 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011503800000000001 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611693+0000", 2024-04-18T23:12:00.567 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9270000000000002e-06 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.713563+0000", 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101869986 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.713746+0000", 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018268700000000001 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:45.019166+0000", 2024-04-18T23:12:00.568 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30542016300000002 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:638 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~114688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:44.611596+0000", 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 15.944124943, 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.45990405099999998, 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.569 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 638 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611596+0000", 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611596+0000", 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.570 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611596+0000", 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6600000000000003e-07 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611598+0000", 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1330000000000002e-06 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611603+0000", 2024-04-18T23:12:00.571 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5480000000000003e-06 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611608+0000", 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7389999999999999e-06 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.614722+0000", 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031144559999999998 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.614728+0000", 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2729999999999999e-06 2024-04-18T23:12:00.572 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.814553+0000", 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19982428199999999 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.814741+0000", 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018874699999999999 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:45.071500+0000", 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.25675860700000003 2024-04-18T23:12:00.573 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.2s4 82/67 ECSubWrite(tid=230, reqid=client.4568.0:665, at_version=82'53, trim_to=0'0, roll_forward_to=81'51))", 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:48.669870+0000", 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 11.885851057, 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30596547600000001, 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.669870+0000", 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.669870+0000", 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.669870+0000", 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7700000000000005e-07 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.670549+0000", 2024-04-18T23:12:00.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00067899300000000002 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.670592+0000", 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.2568999999999998e-05 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.670616+0000", 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4448999999999999e-05 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.971588+0000", 2024-04-18T23:12:00.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30097203500000003 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.971607+0000", 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8839999999999999e-05 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.975770+0000", 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0041625150000000003 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:48.975835+0000", 2024-04-18T23:12:00.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5598000000000005e-05 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:700 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~0] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.839315+0000", 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.716406295000001, 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60877221999999998, 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.578 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 700 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839315+0000", 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839315+0000", 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.579 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839318+0000", 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.286e-06 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839333+0000", 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4970000000000001e-05 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839370+0000", 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7311999999999998e-05 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839395+0000", 2024-04-18T23:12:00.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5026000000000001e-05 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839543+0000", 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014795699999999999 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839731+0000", 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018835799999999999 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.841353+0000", 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0016218070000000001 2024-04-18T23:12:00.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.846909+0000", 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0055556499999999997 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.447943+0000", 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60103379700000004 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.448087+0000", 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014405700000000001 2024-04-18T23:12:00.582 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:701 3.5s0 3:a6fe874e:::smithi14230322-33:head [rollback 1b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.839449+0000", 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.716272029000001, 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.72747437500000001, 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 701 2024-04-18T23:12:00.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839449+0000", 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839449+0000", 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839450+0000", 2024-04-18T23:12:00.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.677e-06 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839460+0000", 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5030000000000003e-06 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839481+0000", 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1359999999999999e-05 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.585 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839499+0000", 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7549000000000001e-05 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.842939+0000", 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003440425 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.843157+0000", 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021817200000000001 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.844478+0000", 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001320697 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.846923+0000", 2024-04-18T23:12:00.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024451770000000002 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566709+0000", 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71978546300000001 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566923+0000", 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021435200000000001 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.588 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:702 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~0] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.839557+0000", 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.716163461000001, 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.72777091400000005, 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 702 2024-04-18T23:12:00.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839557+0000", 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839557+0000", 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839559+0000", 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.24e-06 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839565+0000", 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5830000000000003e-06 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839586+0000", 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0455999999999999e-05 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839608+0000", 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2578999999999998e-05 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.845588+0000", 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0059799830000000003 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.845742+0000", 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000154096 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.846498+0000", 2024-04-18T23:12:00.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00075576500000000004 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.847239+0000", 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00074123500000000003 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.567154+0000", 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71991484800000005 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.567328+0000", 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000174129 2024-04-18T23:12:00.593 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:716 3.5s0 3:a832c25e:::smithi14230322-46:head [append 0~753664] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.944077+0000", 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.611644314999999, 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.63339562400000005, 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.594 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 716 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.944077+0000", 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.944077+0000", 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.944079+0000", 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1770000000000001e-06 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945290+0000", 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012111699999999999 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945320+0000", 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0173000000000001e-05 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945335+0000", 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4909000000000001e-05 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945516+0000", 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018068899999999999 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945705+0000", 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000189829 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.947796+0000", 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002090728 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.952820+0000", 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0050237529999999997 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.577336+0000", 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.624516352 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.577472+0000", 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013584400000000001 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:717 3.5s0 3:a832c25e:::smithi14230322-46:head [append 0~688128] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.945360+0000", 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.610360366, 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.63253810799999999, 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.599 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 717 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945360+0000", 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945360+0000", 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945362+0000", 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2860000000000001e-06 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946063+0000", 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00070088599999999996 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946077+0000", 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4793999999999999e-05 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.601 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946087+0000", 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7759999999999992e-06 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.949388+0000", 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033004850000000001 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.949576+0000", 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000188412 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.951394+0000", 2024-04-18T23:12:00.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001817962 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.055973+0000", 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10457884000000001 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.577726+0000", 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.52175303799999995 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.577899+0000", 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017262899999999999 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.603 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:718 3.5s0 3:a832c25e:::smithi14230322-46:head [append 0~475136] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.946109+0000", 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.609611865, 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.63219438400000005, 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 718 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.604 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946109+0000", 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946109+0000", 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946110+0000", 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.540000000000001e-07 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946595+0000", 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00048526400000000001 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946611+0000", 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6099999999999998e-05 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046764+0000", 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100152771 2024-04-18T23:12:00.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.053013+0000", 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0062488439999999999 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.053213+0000", 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020024299999999999 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.054693+0000", 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001479408 2024-04-18T23:12:00.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.056313+0000", 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001619948 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.578170+0000", 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.521857389 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.578303+0000", 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013346300000000001 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:719 3.5s0 3:a832c25e:::smithi14230322-46:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:50.046818+0000", 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.508902641000001, 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.63283090600000003, 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.609 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 719 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046818+0000", 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046818+0000", 2024-04-18T23:12:00.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046820+0000", 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1900000000000002e-06 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046838+0000", 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7379000000000001e-05 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046864+0000", 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5803e-05 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147013+0000", 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100149482 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147193+0000", 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000180309 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147532+0000", 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00033901800000000001 2024-04-18T23:12:00.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.148946+0000", 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014138499999999999 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.447622+0000", 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.298675886 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.679477+0000", 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.23185497699999999 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.679649+0000", 2024-04-18T23:12:00.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017201200000000001 2024-04-18T23:12:00.614 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.614 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.614 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.614 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:720 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:00.615 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:50.147067+0000", 2024-04-18T23:12:00.615 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.408654282000001, 2024-04-18T23:12:00.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.91909054300000004, 2024-04-18T23:12:00.615 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.615 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.615 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 720 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147067+0000", 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147067+0000", 2024-04-18T23:12:00.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147069+0000", 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.063e-06 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147081+0000", 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-05 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147103+0000", 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2436e-05 2024-04-18T23:12:00.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147120+0000", 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7010000000000001e-05 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.149640+0000", 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025200600000000002 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.149905+0000", 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026456 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.679775+0000", 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.52987033900000002 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.679978+0000", 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000202812 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:51.066157+0000", 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.386179363 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:793 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~425984] snapc 1d=[1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:54.694912+0000", 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.8608090400000004, 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31074624499999998, 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.620 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 793 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.694912+0000", 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.694912+0000", 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.694914+0000", 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.057e-06 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.695337+0000", 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00042273300000000001 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.695365+0000", 2024-04-18T23:12:00.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8393000000000001e-05 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.695380+0000", 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5259000000000001e-05 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.699622+0000", 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042416279999999999 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.699786+0000", 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016424300000000001 2024-04-18T23:12:00.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.701170+0000", 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0013842819999999999 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.798268+0000", 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.097097223999999996 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.005523+0000", 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.207255255 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.005658+0000", 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013517100000000001 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:794 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~442368] snapc 1d=[1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:54.698898+0000", 2024-04-18T23:12:00.625 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.8568223829999999, 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30786606700000002, 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 794 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.698898+0000", 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.698898+0000", 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.698901+0000", 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3910000000000001e-06 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.699349+0000", 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00044807700000000001 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.699374+0000", 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5097e-05 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.699389+0000", 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5423000000000001e-05 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.795323+0000", 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.095933471000000006 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.795512+0000", 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018947500000000001 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.796914+0000", 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001401729 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.798812+0000", 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001898269 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.006539+0000", 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20772634000000001 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.006765+0000", 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022579499999999999 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:795 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~540672] snapc 1d=[1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:54.703975+0000", 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.8517459160000005, 2024-04-18T23:12:00.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30322147399999999, 2024-04-18T23:12:00.634 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.634 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.634 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.634 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.634 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.634 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.634 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 795 2024-04-18T23:12:00.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.634 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.703975+0000", 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.703975+0000", 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.703976+0000", 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.381e-06 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.704478+0000", 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000501776 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.704500+0000", 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1994e-05 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.704514+0000", 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4032999999999999e-05 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.899682+0000", 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19516741700000001 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.900013+0000", 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000331322 2024-04-18T23:12:00.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.902091+0000", 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002078599 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.904937+0000", 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0028457389999999999 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.638 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.007042+0000", 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102104426 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.007196+0000", 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015478700000000001 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:796 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~147456] snapc 1d=[1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:54.705423+0000", 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.8502982289999998, 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50985007800000004, 2024-04-18T23:12:00.639 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 796 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705423+0000", 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705423+0000", 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705424+0000", 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1260000000000001e-06 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705559+0000", 2024-04-18T23:12:00.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013499399999999999 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705576+0000", 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7465999999999998e-05 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705584+0000", 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0399999999999993e-06 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.008071+0000", 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30248644499999999 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.008371+0000", 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029988999999999998 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.010226+0000", 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001855754 2024-04-18T23:12:00.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013512+0000", 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032859149999999999 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.215049+0000", 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20153676700000001 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.215273+0000", 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022368099999999999 2024-04-18T23:12:00.644 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:797 3.5s0 3:a6fe874e:::smithi14230322-33:head [setxattr _header (57) in=7b] snapc 1d=[1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:54.705597+0000", 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.8501236480000003, 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61118310200000003, 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.645 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 797 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705597+0000", 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705597+0000", 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705598+0000", 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9299999999999997e-07 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705601+0000", 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.332e-06 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705607+0000", 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2230000000000004e-06 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705613+0000", 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.13e-06 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.011256+0000", 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30564371299999998 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.011440+0000", 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018343 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:00.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.012418+0000", 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00097794900000000009 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013799+0000", 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001381506 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.316552+0000", 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.302752888 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.316780+0000", 2024-04-18T23:12:00.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022823799999999999 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:798 3.5s0 3:a6fe874e:::smithi14230322-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e88)", 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:54.705619+0000", 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.8501018069999997, 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71576949899999998, 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:00.650 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 798 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705619+0000", 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705619+0000", 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705619+0000", 2024-04-18T23:12:00.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1300000000000001e-07 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705622+0000", 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0870000000000001e-06 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705629+0000", 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.418e-06 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705634+0000", 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6550000000000003e-06 2024-04-18T23:12:00.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.317848+0000", 2024-04-18T23:12:00.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61221415199999996 2024-04-18T23:12:00.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:00.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.318138+0000", 2024-04-18T23:12:00.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029039700000000002 2024-04-18T23:12:00.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.654 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.421389+0000", 2024-04-18T23:12:00.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103250377 2024-04-18T23:12:00.655 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.655 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.655 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.655 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 88/72 ECSubWriteReply(tid=68, last_complete=88'117, committed=1, applied=1))", 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:55.013858+0000", 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.5418625089999995, 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30177705100000002, 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013858+0000", 2024-04-18T23:12:00.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013858+0000", 2024-04-18T23:12:00.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013860+0000", 2024-04-18T23:12:00.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.486e-06 2024-04-18T23:12:00.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013868+0000", 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.6370000000000005e-06 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013896+0000", 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7149e-05 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013909+0000", 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3149999999999999e-05 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.315573+0000", 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30166380100000001 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.315635+0000", 2024-04-18T23:12:00.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2828000000000002e-05 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.5s0 88/72 ECSubWriteReply(tid=68, last_complete=88'117, committed=1, applied=1))", 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:55.015063+0000", 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 5.5406573620000001, 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30103142799999999, 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.015063+0000", 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.015063+0000", 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.015065+0000", 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.203e-06 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.015073+0000", 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0660000000000004e-06 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.015095+0000", 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2283e-05 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.015105+0000", 2024-04-18T23:12:00.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0154e-05 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.316036+0000", 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30093059999999999 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.316095+0000", 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9122000000000001e-05 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:00.664 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:00.666 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:12:00.667 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:00.662+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:00.767 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:00.762+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs1 94/31 ECSubWrite(tid=408, reqid=client.4568.0:828, at_version=94'170, trim_to=0'0, roll_forward_to=92'163))", 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.760983+0000", 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.039149377999999999, 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039192762999999999, 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "no flag points reached", 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:00.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.760983+0000", 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.760983+0000", 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.760984+0000", 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1799999999999999e-06 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761061+0000", 2024-04-18T23:12:00.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6301e-05 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761085+0000", 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4596999999999998e-05 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 1 2024-04-18T23:12:00.802 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:00.814 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:12:00.868 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:00.867+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:00.964 INFO:tasks.rados.rados.0.smithi142.stdout:334: finishing write tid 1 to smithi14230322-45 2024-04-18T23:12:00.964 INFO:tasks.rados.rados.0.smithi142.stdout:334: finishing write tid 2 to smithi14230322-45 2024-04-18T23:12:00.969 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:00.966+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:00.982 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:00.982 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:12:00.982 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:12:00.982 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:12:00.982 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:00.996 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:12:01.070 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:01.068+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:01.168 INFO:tasks.rados.rados.0.smithi142.stdout:334: finishing write tid 3 to smithi14230322-45 2024-04-18T23:12:01.169 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:01.169 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:12:01.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:12:01.169 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:12:01.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.169 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:417 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~524288 in=524288b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:01.169 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:12:01.169 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 39.645915778999999, 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30835631899999999, 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 417 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520104+0000", 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2389999999999999e-06 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520670+0000", 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00056593599999999996 2024-04-18T23:12:01.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520690+0000", 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9945e-05 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520699+0000", 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0429999999999996e-06 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618217+0000", 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097518176999999998 2024-04-18T23:12:01.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618227+0000", 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7860000000000008e-06 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622364+0000", 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041371389999999997 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622653+0000", 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028858600000000002 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.624345+0000", 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001691782 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727165+0000", 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102820087 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828093+0000", 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100928079 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828459+0000", 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036652000000000001 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:418 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~245760 in=245760b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 39.644432922999997, 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30828365299999999, 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.175 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 418 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521587+0000", 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2100000000000001e-06 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521960+0000", 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000373402 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521972+0000", 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.207e-05 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521979+0000", 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7839999999999998e-06 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725251+0000", 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20327163200000001 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725428+0000", 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000177071 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.726423+0000", 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00099555200000000003 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828646+0000", 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102223007 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829746+0000", 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010997509999999999 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829869+0000", 2024-04-18T23:12:01.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012317400000000001 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:419 3.6s0 3:635103ff:::smithi14230322-31:head [setxattr _header (57) in=7b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 39.644027905999998, 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71019062499999996, 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.180 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 419 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2200000000000004e-07 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521994+0000", 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8289999999999999e-06 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521999+0000", 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8470000000000003e-06 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522003+0000", 2024-04-18T23:12:01.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1250000000000003e-06 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828718+0000", 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30671521899999998 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828796+0000", 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7502000000000004e-05 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829209+0000", 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041271100000000001 2024-04-18T23:12:01.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829944+0000", 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00073506600000000004 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.231940+0000", 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.401996146 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232181+0000", 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024165800000000001 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.184 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:420 3.6s0 3:635103ff:::smithi14230322-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 39.644010233000003, 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.108457993, 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 420 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.185 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522009+0000", 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4999999999999998e-07 2024-04-18T23:12:01.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522012+0000", 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.819e-06 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522017+0000", 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2100000000000001e-06 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522021+0000", 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1400000000000002e-06 2024-04-18T23:12:01.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829499+0000", 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30747766399999998 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829621+0000", 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000122502 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232655+0000", 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.403033435 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232821+0000", 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000166279 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.630466+0000", 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39764559399999999 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(osd.7.22:3 3.6s0 3:635103ff:::smithi14230322-31:head [assert-version v20,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:12:01.189 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:31.831327+0000", 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 29.334691775, 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.21208626, 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831327+0000", 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831327+0000", 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831329+0000", 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2929999999999999e-06 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831339+0000", 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7000000000000003e-06 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831359+0000", 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9743000000000001e-05 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.831373+0000", 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4583000000000001e-05 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.935084+0000", 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103711043 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:31.935260+0000", 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00017573299999999999 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:32.043413+0000", 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.108153165 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.4s4 81/31 ECSubWrite(tid=110, reqid=osd.8.0:110, at_version=81'66, trim_to=0'0, roll_forward_to=80'64))", 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:47.640383+0000", 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.525635572000001, 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.223413846, 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.193 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.640383+0000", 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.640383+0000", 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.194 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.640383+0000", 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6300000000000001e-07 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.640410+0000", 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6798999999999999e-05 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.640439+0000", 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8795000000000001e-05 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.640451+0000", 2024-04-18T23:12:01.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2213e-05 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.763714+0000", 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.12326266399999999 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.763728+0000", 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4618e-05 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.863755+0000", 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100026199 2024-04-18T23:12:01.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.863797+0000", 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2295000000000001e-05 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.4s4 81/31 ECSubWrite(tid=111, reqid=osd.8.0:111, at_version=81'68, trim_to=0'0, roll_forward_to=80'64))", 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:47.641536+0000", 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 13.524482624999999, 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.22228840699999999, 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:01.197 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.641536+0000", 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.641536+0000", 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.641537+0000", 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3900000000000004e-07 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.641560+0000", 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2921e-05 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.641575+0000", 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5586999999999998e-05 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.641586+0000", 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0856000000000001e-05 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.764181+0000", 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.12259458600000001 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.764193+0000", 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2762000000000001e-05 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.863803+0000", 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099609764000000003 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:47.863824+0000", 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1291999999999999e-05 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:721 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~770048] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.882464+0000", 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.283554189, 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.37029923799999998, 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.201 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 721 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.882464+0000", 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.882464+0000", 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.882467+0000", 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6460000000000002e-06 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883686+0000", 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012187159999999999 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883718+0000", 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2186e-05 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883733+0000", 2024-04-18T23:12:01.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4970000000000001e-05 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883823+0000", 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0488999999999994e-05 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883966+0000", 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014226500000000001 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.885426+0000", 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014602409999999999 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.890445+0000", 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0050192099999999996 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.252676+0000", 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.36223060800000001 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.252764+0000", 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7906999999999997e-05 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.205 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:722 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~704512] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.887144+0000", 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.278874728, 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38817684699999999, 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 722 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.206 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.887144+0000", 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.887144+0000", 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.887153+0000", 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0799999999999995e-06 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888445+0000", 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00129243 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888476+0000", 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0586000000000002e-05 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888492+0000", 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5933000000000001e-05 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888562+0000", 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9862000000000003e-05 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888675+0000", 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113402 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.889711+0000", 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001035869 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.995104+0000", 2024-04-18T23:12:01.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.105393192 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275227+0000", 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.28012296199999998 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275321+0000", 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3530999999999996e-05 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:723 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~458752] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.890673+0000", 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.275345755, 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38502245800000001, 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 723 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.890673+0000", 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.890673+0000", 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.890675+0000", 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.753e-06 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891295+0000", 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00062035499999999995 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891321+0000", 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5469e-05 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891333+0000", 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2816999999999999e-05 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891402+0000", 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9068999999999999e-05 2024-04-18T23:12:01.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891521+0000", 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000118157 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.892362+0000", 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00084165300000000004 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.995106+0000", 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102743341 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275555+0000", 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.28044971400000002 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275695+0000", 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014013 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:724 3.6s0 3:635103ff:::smithi14230322-31:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891361+0000", 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.274657632, 2024-04-18T23:12:01.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38458247699999998, 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 724 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891361+0000", 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891361+0000", 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891362+0000", 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.387e-06 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891370+0000", 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2860000000000002e-06 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891389+0000", 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9108999999999999e-05 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891404+0000", 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5024e-05 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.892942+0000", 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001537957 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.893044+0000", 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010249799999999999 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.893555+0000", 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051112599999999998 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.995108+0000", 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101552937 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275846+0000", 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.28073780100000001 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275944+0000", 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7351999999999999e-05 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:725 3.6s0 3:635103ff:::smithi14230322-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891424+0000", 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.274594713999999, 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.68865348800000004, 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.220 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 725 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891424+0000", 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891424+0000", 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891426+0000", 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7370000000000001e-06 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891433+0000", 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3459999999999996e-06 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891452+0000", 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8972999999999999e-05 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891466+0000", 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3852999999999999e-05 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.893797+0000", 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0023310200000000001 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.893902+0000", 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104888 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.276479+0000", 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38257698400000001 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.276775+0000", 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000296228 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.580077+0000", 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30330245900000002 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:726 3.6s0 3:6db21afe:::smithi14230322-12:15 [read 0~131072,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891478+0000", 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.274540291999999, 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38608822199999998, 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 726 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891478+0000", 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891478+0000", 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891479+0000", 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9699999999999995e-07 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891484+0000", 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7469999999999996e-06 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891493+0000", 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3160000000000003e-06 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891500+0000", 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1860000000000003e-06 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.894032+0000", 2024-04-18T23:12:01.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025314819999999998 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.894127+0000", 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4858000000000006e-05 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.277567+0000", 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38343993599999998 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:727 3.6s0 3:6db21afe:::smithi14230322-12:15 [read 0~131072,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.228 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891510+0000", 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.274508948999999, 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.48656820899999997, 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 727 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891510+0000", 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891510+0000", 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891510+0000", 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0299999999999999e-07 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891514+0000", 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4070000000000002e-06 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891522+0000", 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6459999999999999e-06 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891529+0000", 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8929999999999999e-06 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.994525+0000", 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10299591600000001 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.994591+0000", 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6243999999999998e-05 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.378078+0000", 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38348650000000001 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:728 3.6s0 3:6db21afe:::smithi14230322-12:15 [read 0~131072,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891537+0000", 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.274481577, 2024-04-18T23:12:01.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.58775005700000005, 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 728 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891537+0000", 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891537+0000", 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891538+0000", 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9999999999999997e-07 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891542+0000", 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9330000000000003e-06 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891550+0000", 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2870000000000004e-06 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891557+0000", 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9870000000000002e-06 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.994791+0000", 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10323368600000001 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.994865+0000", 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4899000000000002e-05 2024-04-18T23:12:01.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.479287+0000", 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.484421665 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.5s5 83/72 ECSubWrite(tid=39, reqid=client.4568.0:701, at_version=83'87, trim_to=0'0, roll_forward_to=82'84))", 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.990703+0000", 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.175315841, 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30279335000000002, 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.236 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990703+0000", 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990703+0000", 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990703+0000", 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5300000000000004e-07 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990722+0000", 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8445999999999999e-05 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990737+0000", 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.47e-05 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990748+0000", 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1756000000000001e-05 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.090937+0000", 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10018898900000001 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.090953+0000", 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6079999999999999e-05 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293414+0000", 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20246087500000001 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293496+0000", 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1850999999999999e-05 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.5s5 83/72 ECSubWrite(tid=40, reqid=client.4568.0:702, at_version=83'88, trim_to=0'0, roll_forward_to=82'84))", 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.990770+0000", 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.175248213, 2024-04-18T23:12:01.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30279787000000002, 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990770+0000", 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990770+0000", 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990771+0000", 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3000000000000001e-07 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990803+0000", 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2353e-05 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990820+0000", 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6929e-05 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990829+0000", 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7600000000000008e-06 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.091673+0000", 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100843842 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.091704+0000", 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1582999999999999e-05 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293505+0000", 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20180018199999999 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293568+0000", 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3690999999999997e-05 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds4 83/31 ECSubWrite(tid=142, reqid=client.4568.0:705, at_version=83'120, trim_to=0'0, roll_forward_to=83'116))", 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.060665+0000", 2024-04-18T23:12:01.244 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.105353245, 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.232972237, 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.060665+0000", 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.060665+0000", 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.060667+0000", 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.933e-06 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.061042+0000", 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00037508399999999998 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.061083+0000", 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0286e-05 2024-04-18T23:12:01.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.061119+0000", 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6001999999999999e-05 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.091613+0000", 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03049437 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.091637+0000", 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4162e-05 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293573+0000", 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20193602499999999 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293638+0000", 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4375000000000006e-05 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds4 83/31 ECSubWrite(tid=143, reqid=client.4568.0:706, at_version=83'121, trim_to=0'0, roll_forward_to=83'116))", 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.062750+0000", 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.103268669, 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.23127808799999999, 2024-04-18T23:12:01.248 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.062750+0000", 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.062750+0000", 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.062752+0000", 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5540000000000001e-06 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.062782+0000", 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0131999999999999e-05 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.062812+0000", 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.048e-05 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.062835+0000", 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2665999999999999e-05 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.292832+0000", 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.229996967 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.292849+0000", 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7051999999999999e-05 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.251 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293981+0000", 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011325280000000001 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.294028+0000", 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6709000000000003e-05 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpReadReply(3.6s0 83/72 ECSubReadReply(tid=56, attrs_read=0))", 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.969421993999999, 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.28271478999999999, 2024-04-18T23:12:01.252 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:01.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.296432+0000", 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099835462 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.396590+0000", 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10015741 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.478986+0000", 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.082396805000000004 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.479311+0000", 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00032511299999999998 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.255 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:01.257 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:12:01.257 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:01.255+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:01.269 INFO:tasks.rados.rados.0.smithi142.stdout:334: finishing write tid 4 to smithi14230322-45 2024-04-18T23:12:01.269 INFO:tasks.rados.rados.0.smithi142.stdout:334: finishing write tid 5 to smithi14230322-45 2024-04-18T23:12:01.271 INFO:tasks.rados.rados.0.smithi142.stdout:335: finishing rollback tid 0 to smithi14230322-35 2024-04-18T23:12:01.358 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:01.356+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:01.378 INFO:tasks.rados.rados.0.smithi142.stdout:334: finishing write tid 6 to smithi14230322-45 2024-04-18T23:12:01.378 INFO:tasks.rados.rados.0.smithi142.stdout:334: oid 45 updating version 0 to 169 2024-04-18T23:12:01.378 INFO:tasks.rados.rados.0.smithi142.stdout:334: oid 45 updating version 169 to 171 2024-04-18T23:12:01.378 INFO:tasks.rados.rados.0.smithi142.stdout:334: oid 45 version 171 is already newer than 165 2024-04-18T23:12:01.378 INFO:tasks.rados.rados.0.smithi142.stdout:334: oid 45 version 171 is already newer than 168 2024-04-18T23:12:01.378 INFO:tasks.rados.rados.0.smithi142.stdout:334: oid 45 version 171 is already newer than 170 2024-04-18T23:12:01.378 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 45 v 171 (ObjNum 124 snap 32 seq_num 124) dirty exists 2024-04-18T23:12:01.378 INFO:tasks.rados.rados.0.smithi142.stdout:334: left oid 45 (ObjNum 124 snap 32 seq_num 124) 2024-04-18T23:12:01.385 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:01.385 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:12:01.385 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.385 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.bs0 94/31 ECSubWriteReply(tid=414, last_complete=94'175, committed=1, applied=1))", 2024-04-18T23:12:01.385 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.378634+0000", 2024-04-18T23:12:01.385 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.0067890199999999998, 2024-04-18T23:12:01.385 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0068252490000000002, 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "no flag points reached", 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.378634+0000", 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.378634+0000", 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.386 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.378635+0000", 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4959999999999999e-06 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.378644+0000", 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8200000000000003e-06 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.378665+0000", 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1075999999999999e-05 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.387 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:832 3.bs0 3:d92c4d07:::smithi14230322-35:head [rollback 13] snapc 21=[21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.557548+0000", 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.82787535099999998, 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82800341399999999, 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.388 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 832 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557548+0000", 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557548+0000", 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.389 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557548+0000", 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3300000000000002e-07 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557551+0000", 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.655e-06 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557561+0000", 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7200000000000001e-06 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.390 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557567+0000", 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5529999999999998e-06 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.272507+0000", 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71493971000000001 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.272706+0000", 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000199175 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.391 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.273798+0000", 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001091809 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.275490+0000", 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00169257 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:833 3.bs0 3:d92c4d07:::smithi14230322-35:head [append 0~0] snapc 21=[21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.557575+0000", 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.82784755600000004, 2024-04-18T23:12:01.392 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82811630599999997, 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 833 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557575+0000", 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.393 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557575+0000", 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557576+0000", 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7400000000000003e-07 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557579+0000", 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6089999999999999e-06 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.394 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557587+0000", 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9850000000000006e-06 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557593+0000", 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2790000000000004e-06 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.275548+0000", 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71795553899999998 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.395 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.275707+0000", 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015863300000000001 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.276448+0000", 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000741385 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.277545+0000", 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010963139999999999 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.396 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.397 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-18T23:12:01.397 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 3 2024-04-18T23:12:01.397 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:01.401 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:12:01.458 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:01.454+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:01.474 INFO:tasks.rados.rados.0.smithi142.stdout:335: finishing rollback tid 1 to smithi14230322-35 2024-04-18T23:12:01.478 INFO:tasks.rados.rados.0.smithi142.stdout:335: finishing rollback tid 2 to smithi14230322-35 2024-04-18T23:12:01.478 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 35 v 175 (ObjNum 75 snap 13 seq_num 75) dirty exists 2024-04-18T23:12:01.526 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:01.526 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:12:01.526 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:12:01.526 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:12:01.526 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:01.536 INFO:tasks.rados.rados.0.smithi142.stdout:332: done (3 left) 2024-04-18T23:12:01.536 INFO:tasks.rados.rados.0.smithi142.stdout:334: done (2 left) 2024-04-18T23:12:01.536 INFO:tasks.rados.rados.0.smithi142.stdout:335: done (1 left) 2024-04-18T23:12:01.536 INFO:tasks.rados.rados.0.smithi142.stdout:336: done (0 left) 2024-04-18T23:12:01.536 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 18 current snap is 33 2024-04-18T23:12:01.536 INFO:tasks.rados.rados.0.smithi142.stdout:337: seq_num 125 ranges {4669440=557056,5226496=475136,5701632=507904,6209536=475136,6684672=98304} 2024-04-18T23:12:01.542 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:12:01.559 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:01.556+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:01.570 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:12:01.588 INFO:tasks.rados.rados.0.smithi142.stdout:337: writing smithi14230322-18 from 4669440 to 5226496 tid 1 2024-04-18T23:12:01.593 INFO:tasks.rados.rados.0.smithi142.stdout:337: writing smithi14230322-18 from 5226496 to 5701632 tid 2 2024-04-18T23:12:01.597 INFO:tasks.rados.rados.0.smithi142.stdout:337: writing smithi14230322-18 from 5701632 to 6209536 tid 3 2024-04-18T23:12:01.601 INFO:tasks.rados.rados.0.smithi142.stdout:337: writing smithi14230322-18 from 6209536 to 6684672 tid 4 2024-04-18T23:12:01.603 INFO:tasks.rados.rados.0.smithi142.stdout:337: writing smithi14230322-18 from 6684672 to 6782976 tid 5 2024-04-18T23:12:01.603 INFO:tasks.rados.rados.0.smithi142.stdout:338: read oid 31 snap -1 2024-04-18T23:12:01.603 INFO:tasks.rados.rados.0.smithi142.stdout:338: expect (ObjNum 115 snap 26 seq_num 115) 2024-04-18T23:12:01.603 INFO:tasks.rados.rados.0.smithi142.stdout:339: read oid 5 snap -1 2024-04-18T23:12:01.603 INFO:tasks.rados.rados.0.smithi142.stdout:339: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-18T23:12:01.603 INFO:tasks.rados.rados.0.smithi142.stdout:340: copy_from oid 21 from oid 32 current snap is 33 2024-04-18T23:12:01.603 INFO:tasks.rados.rados.0.smithi142.stdout:341: snap_create 2024-04-18T23:12:01.660 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:01.660+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:184 3.bs0 3:dd01ba5f:::smithi14230322-23:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.109001970000001, 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61588253999999998, 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.668 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 184 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557491+0000", 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8700000000000001e-07 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557493+0000", 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9149999999999999e-06 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557499+0000", 2024-04-18T23:12:01.670 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.2650000000000002e-06 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.557504+0000", 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5689999999999998e-06 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654492+0000", 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.096987633000000004 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.654498+0000", 2024-04-18T23:12:01.671 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5860000000000001e-06 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757767+0000", 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.103269131 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757839+0000", 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1878999999999994e-05 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967626+0000", 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.209786891 2024-04-18T23:12:01.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.967877+0000", 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025035699999999999 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173373+0000", 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.205496927 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:189 3.bs0 3:d498df52:::smithi14230322-7:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:12:01.673 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 61.107809688000003, 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.91155819000000005, 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 189 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.674 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558683+0000", 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558684+0000", 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1799999999999995e-07 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558687+0000", 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1920000000000001e-06 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.675 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558694+0000", 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.452e-06 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.658767+0000", 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100072129 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757920+0000", 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099153626999999994 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.676 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.757967+0000", 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6962999999999997e-05 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.171274+0000", 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41330696700000003 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.173905+0000", 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002630828 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.677 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470100+0000", 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29619521399999998 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:01.470241+0000", 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014119999999999999 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:279 3.7s0 3:ea1bab24:::smithi14230322-36:head [copy-from ver 15] snapc 6=[6,5,4] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.579501+0000", 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.086991771000001, 2024-04-18T23:12:01.678 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.81417189400000001, 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 279 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579501+0000", 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.679 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579501+0000", 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579503+0000", 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.798e-06 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579508+0000", 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4219999999999999e-06 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579520+0000", 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1833e-05 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.579531+0000", 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1080999999999999e-05 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677050+0000", 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097518821000000006 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.677182+0000", 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013219100000000001 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.091592+0000", 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41440982100000001 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.095981+0000", 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0043885729999999998 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.682 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.393572+0000", 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29759129499999998 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:07.393673+0000", 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000101059 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:605 3.7s0 3:e8848f63:::smithi14230322-47:head [read 0~786432,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:43.599532+0000", 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 18.066960672, 2024-04-18T23:12:01.683 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.632778481, 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 605 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.599532+0000", 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.684 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.599532+0000", 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.599533+0000", 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7000000000000005e-07 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.599536+0000", 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.715e-06 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.599548+0000", 2024-04-18T23:12:01.685 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1157e-05 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.599556+0000", 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9680000000000006e-06 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.599826+0000", 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000270375 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.599844+0000", 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8105e-05 2024-04-18T23:12:01.686 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.798748+0000", 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19890396699999999 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.798798+0000", 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9851000000000001e-05 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.232311+0000", 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.43351277300000002 2024-04-18T23:12:01.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:610 3.bs0 3:deec07be:::smithi14230322-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:43.720796+0000", 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 17.94569641, 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.91799579200000003, 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 610 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.688 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720796+0000", 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720796+0000", 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.689 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720799+0000", 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.199e-06 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720816+0000", 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6857000000000001e-05 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720834+0000", 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8535000000000001e-05 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.690 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720846+0000", 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1749999999999999e-05 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.821968+0000", 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101121849 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.822112+0000", 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000144234 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.691 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.229460+0000", 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40734825400000002 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.229624+0000", 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016394599999999999 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.638792+0000", 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.409168168 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:612 3.bs0 3:d447fef0:::smithi14230322-49:15 [read 0~1900544,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:43.720903+0000", 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 17.945590040999999, 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71460734599999998, 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 612 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.693 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720903+0000", 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720903+0000", 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720904+0000", 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5199999999999995e-07 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.694 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720908+0000", 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3089999999999997e-06 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720918+0000", 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0339999999999999e-05 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720926+0000", 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6340000000000001e-06 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.822919+0000", 2024-04-18T23:12:01.695 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101992896 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.823025+0000", 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000105763 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.435510+0000", 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61248545200000004 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.696 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:613 3.bs0 3:d447fef0:::smithi14230322-49:15 [read 0~1900544,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:43.720936+0000", 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 17.945557083000001, 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.81682598699999998, 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 613 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720936+0000", 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.697 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720936+0000", 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720936+0000", 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.06e-07 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720951+0000", 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4783999999999999e-05 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.698 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720959+0000", 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2679999999999994e-06 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.720967+0000", 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4000000000000003e-06 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.824704+0000", 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10373755699999999 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.699 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:43.824836+0000", 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013204800000000001 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:44.537762+0000", 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71292532399999997 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:696 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~802816] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.834553+0000", 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.831940112, 2024-04-18T23:12:01.700 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82256638900000001, 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 696 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.834553+0000", 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.701 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.834553+0000", 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.834555+0000", 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.553e-06 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.835595+0000", 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00104009 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.835623+0000", 2024-04-18T23:12:01.702 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7722999999999999e-05 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.835640+0000", 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7016000000000001e-05 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.932100+0000", 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.096459767000000002 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.932224+0000", 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012372699999999999 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.703 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.933395+0000", 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011709019999999999 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.934862+0000", 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014671160000000001 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.657005+0000", 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.72214325300000004 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.704 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.657119+0000", 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00011424199999999999 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:697 3.bs0 3:d447fef0:::smithi14230322-49:head [append 0~360448] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.838168+0000", 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.828325051, 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.81920591899999995, 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.705 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 697 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.838168+0000", 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.838168+0000", 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.706 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.838170+0000", 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1430000000000001e-06 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.838650+0000", 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00047960400000000001 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.838680+0000", 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0849999999999998e-05 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.707 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938812+0000", 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10013180100000001 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938887+0000", 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5192999999999996e-05 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938990+0000", 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000103046 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.940110+0000", 2024-04-18T23:12:01.708 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011195619999999999 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.941921+0000", 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00181106 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.657286+0000", 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71536535899999998 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.657374+0000", 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7300999999999996e-05 2024-04-18T23:12:01.709 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:698 3.bs0 3:d447fef0:::smithi14230322-49:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.938848+0000", 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.72764476, 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71884433700000006, 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.710 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 698 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938848+0000", 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938848+0000", 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938850+0000", 2024-04-18T23:12:01.711 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.866e-06 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938860+0000", 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0139999999999999e-05 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938878+0000", 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8213e-05 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938889+0000", 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0776e-05 2024-04-18T23:12:01.712 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.940701+0000", 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001811509 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.940799+0000", 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8473000000000004e-05 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.941291+0000", 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00049214799999999998 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.713 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.942583+0000", 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001291302 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.657520+0000", 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.714937771 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.657692+0000", 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000172139 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.714 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:699 3.bs0 3:d447fef0:::smithi14230322-49:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.938905+0000", 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.727587579, 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.122451367, 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 699 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938905+0000", 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.715 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938905+0000", 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938906+0000", 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6400000000000001e-07 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938910+0000", 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6859999999999999e-06 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.716 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938921+0000", 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1080000000000001e-05 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.938929+0000", 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9500000000000001e-06 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.941612+0000", 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026831609999999999 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.941880+0000", 2024-04-18T23:12:01.717 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00026840300000000001 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.658285+0000", 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71640496899999995 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.658403+0000", 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117865 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:51.061357+0000", 2024-04-18T23:12:01.718 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.402953389 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:730 3.7s0 3:e2ccd2e3:::smithi14230322-38:head [append 0~458752] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.040870+0000", 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.625622807999999, 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61723233200000005, 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.719 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 730 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.040870+0000", 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.040870+0000", 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.040872+0000", 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.66e-06 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.041497+0000", 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00062540999999999996 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.041517+0000", 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.959e-05 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.041533+0000", 2024-04-18T23:12:01.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6266e-05 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.146936+0000", 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10540305899999999 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.147275+0000", 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00033921199999999998 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.149655+0000", 2024-04-18T23:12:01.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002379576 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.155365+0000", 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0057102439999999997 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.658012+0000", 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50264718500000005 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.658102+0000", 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0130000000000006e-05 2024-04-18T23:12:01.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:731 3.7s0 3:e2ccd2e3:::smithi14230322-38:head [append 0~442368] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.041563+0000", 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.624929662, 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61682522200000001, 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 731 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.724 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.041563+0000", 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.041563+0000", 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.041565+0000", 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8500000000000001e-06 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.042350+0000", 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000785083 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.042374+0000", 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3751999999999999e-05 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.042391+0000", 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7e-05 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.150902+0000", 2024-04-18T23:12:01.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10851158299999999 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.151212+0000", 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000309446 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.153454+0000", 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002242506 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.158767+0000", 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0053130640000000002 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.658311+0000", 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.49954337300000001 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.658388+0000", 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7564999999999993e-05 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:732 3.7s0 3:e2ccd2e3:::smithi14230322-38:head [append 0~262144] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.042469+0000", 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.624023759, 2024-04-18T23:12:01.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61616052300000002, 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 732 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.042469+0000", 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.042469+0000", 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.042471+0000", 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6330000000000001e-06 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.042958+0000", 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00048762099999999999 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.042980+0000", 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2013000000000001e-05 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.042996+0000", 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6073e-05 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.155717+0000", 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.112720159 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.156011+0000", 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029455600000000002 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.157835+0000", 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001823383 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.161061+0000", 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003226235 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.658553+0000", 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.497492351 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.658630+0000", 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6499000000000001e-05 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:733 3.7s0 3:e2ccd2e3:::smithi14230322-38:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:01.733 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.043067+0000", 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 11.623426221000001, 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71642073500000003, 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 733 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.043067+0000", 2024-04-18T23:12:01.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.043067+0000", 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.043068+0000", 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.37e-06 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.043078+0000", 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0206e-05 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.043098+0000", 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9826999999999998e-05 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.043114+0000", 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5569e-05 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.158869+0000", 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.11575574900000001 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.159166+0000", 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029633200000000001 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.160106+0000", 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094050600000000002 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.162299+0000", 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021930869999999998 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759234+0000", 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.59693507199999996 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759487+0000", 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025301700000000002 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:827 3.bs0 3:deec07be:::smithi14230322-45:head [append 0~786432] snapc 21=[21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.556451+0000", 2024-04-18T23:12:01.738 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.110041939, 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61220507000000002, 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 827 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.556451+0000", 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.556451+0000", 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.556453+0000", 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8509999999999999e-06 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557318+0000", 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00086569700000000002 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557340+0000", 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1631999999999999e-05 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557352+0000", 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1620000000000001e-05 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.754945+0000", 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19759338600000001 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.755272+0000", 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00032690799999999999 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.757546+0000", 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0022742959999999999 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.968060+0000", 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.21051387799999999 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.168520+0000", 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20045949900000001 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.168656+0000", 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000136303 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:828 3.bs0 3:deec07be:::smithi14230322-45:head [append 0~49152] snapc 21=[21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.557373+0000", 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.109119878, 2024-04-18T23:12:01.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71251337599999998, 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 828 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557373+0000", 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557373+0000", 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557374+0000", 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.47e-07 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557413+0000", 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8955000000000002e-05 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557427+0000", 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4072e-05 2024-04-18T23:12:01.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557435+0000", 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4859999999999997e-06 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.759027+0000", 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201591507 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.759160+0000", 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000133139 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.759851+0000", 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00069085800000000003 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.968062+0000", 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20821070999999999 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.269659+0000", 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30159758399999997 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.269886+0000", 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000227118 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:829 3.bs0 3:deec07be:::smithi14230322-45:head [setxattr _header (57) in=7b] snapc 21=[21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.557448+0000", 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.109044664, 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71278173199999995, 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.748 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 829 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557448+0000", 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557448+0000", 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.749 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557449+0000", 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3399999999999998e-07 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557453+0000", 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1330000000000001e-06 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557466+0000", 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2344000000000001e-05 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557473+0000", 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9440000000000005e-06 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.760698+0000", 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20322462199999999 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.760849+0000", 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015043700000000001 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761416+0000", 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00056758500000000001 2024-04-18T23:12:01.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.968064+0000", 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20664776900000001 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.270139+0000", 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30207535200000002 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.270230+0000", 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0711999999999995e-05 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.752 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:830 3.bs0 3:deec07be:::smithi14230322-45:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e94)", 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.557484+0000", 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.1090087909999999, 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82118848700000002, 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 830 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.753 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557484+0000", 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557484+0000", 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557485+0000", 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1699999999999997e-07 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557488+0000", 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3840000000000001e-06 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557501+0000", 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2493e-05 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557509+0000", 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7749999999999993e-06 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761763+0000", 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20425494 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761868+0000", 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010429099999999999 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.270349+0000", 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50848132300000004 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.270646+0000", 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00029650999999999999 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.378673+0000", 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10802695399999999 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:831 3.bs0 3:d92c4d07:::smithi14230322-35:head [append 0~0] snapc 21=[21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e94)", 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.557519+0000", 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 1.108973961, 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71419477099999995, 2024-04-18T23:12:01.757 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 831 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557519+0000", 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557519+0000", 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557520+0000", 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7800000000000001e-07 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557523+0000", 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.151e-06 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557532+0000", 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1779999999999992e-06 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557539+0000", 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6490000000000002e-06 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.862178+0000", 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30463897699999998 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.862505+0000", 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00032712099999999999 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.864820+0000", 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0023156909999999999 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.168681+0000", 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30386067999999999 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.271646+0000", 2024-04-18T23:12:01.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102964635 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.271714+0000", 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8010999999999996e-05 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.762 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:01.763 INFO:tasks.rados.rados.0.smithi142.stdout:340: finishing copy_from racing read to smithi14230322-21 2024-04-18T23:12:01.763 INFO:tasks.rados.rados.0.smithi142.stdout:340: finishing copy_from to smithi14230322-21 2024-04-18T23:12:01.764 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 21 v 182 (ObjNum 119 snap 26 seq_num 119) dirty exists 2024-04-18T23:12:01.764 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:12:01.765 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:01.758+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:01.811 INFO:tasks.rados.rados.0.smithi142.stdout:339: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-18T23:12:01.849 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:12:01.849 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:12:01.864 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:01.862+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:01.871 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:12:01.871 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:12:01.894 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:12:01.894 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:12:01.918 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:12:01.918 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:12:01.936 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:01.936 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=152, last_complete=95'112, committed=1, applied=1))", 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.849023+0000", 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.086449818999999997, 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.086473979000000006, 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.849023+0000", 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.937 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.849023+0000", 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.849024+0000", 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8699999999999995e-07 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.849027+0000", 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.393e-06 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.938 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.849040+0000", 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2738999999999999e-05 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.849050+0000", 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6500000000000008e-06 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=152, last_complete=95'112, committed=1, applied=1))", 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.748296+0000", 2024-04-18T23:12:01.939 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.18717737500000001, 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.18724581900000001, 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748296+0000", 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748296+0000", 2024-04-18T23:12:01.940 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748296+0000", 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.6100000000000001e-07 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748299+0000", 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0699999999999998e-06 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748307+0000", 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1999999999999997e-06 2024-04-18T23:12:01.941 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848375+0000", 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10006825699999999 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=150, last_complete=95'110, committed=1, applied=1))", 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.748826+0000", 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.18664750699999999, 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.18675339799999999, 2024-04-18T23:12:01.942 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748826+0000", 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748826+0000", 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748826+0000", 2024-04-18T23:12:01.943 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7500000000000001e-07 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748829+0000", 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.531e-06 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748836+0000", 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1230000000000002e-06 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848922+0000", 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008623799999999 2024-04-18T23:12:01.944 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:835 3.4s0 3:2f7b52be:::smithi14230322-18:head [append 0~557056 in=557056b] snapc 22=[22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.589973+0000", 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.34549981200000002, 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.345653559, 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.945 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 835 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.589973+0000", 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.589973+0000", 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.946 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.589975+0000", 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0279999999999999e-06 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.590731+0000", 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00075536199999999998 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.590754+0000", 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3430000000000001e-05 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.590765+0000", 2024-04-18T23:12:01.947 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1151e-05 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.590841+0000", 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5229999999999996e-05 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.590852+0000", 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1270999999999999e-05 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.636725+0000", 2024-04-18T23:12:01.948 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.045872996999999999 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.636819+0000", 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3933999999999999e-05 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.637752+0000", 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00093281500000000003 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.949 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.644267+0000", 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0065155129999999997 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:836 3.4s0 3:2f7b52be:::smithi14230322-18:head [append 0~475136 in=475136b] snapc 22=[22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.594386+0000", 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.34108748500000002, 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.34131921500000001, 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.950 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 836 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.594386+0000", 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.594386+0000", 2024-04-18T23:12:01.951 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.594391+0000", 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.169e-06 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.594769+0000", 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00037827599999999998 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.594786+0000", 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7044000000000001e-05 2024-04-18T23:12:01.952 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.594796+0000", 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4450000000000002e-06 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.594860+0000", 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4412999999999995e-05 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.594865+0000", 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.22e-06 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.953 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.639118+0000", 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.044252832999999998 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.639258+0000", 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013961400000000001 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.640198+0000", 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00094002300000000001 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.954 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.644366+0000", 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004168588 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=152, last_complete=95'112, committed=1, applied=1))", 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.847358+0000", 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.088114734, 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.088413238000000005, 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.955 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847358+0000", 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847358+0000", 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847359+0000", 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.08e-07 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.956 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847362+0000", 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0290000000000001e-06 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847373+0000", 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1527e-05 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847381+0000", 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1369999999999996e-06 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.957 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:837 3.4s0 3:2f7b52be:::smithi14230322-18:head [append 0~507904 in=507904b] snapc 22=[22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.598818+0000", 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.33665519900000002, 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.33699359800000001, 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 837 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.598818+0000", 2024-04-18T23:12:01.958 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.598818+0000", 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.598820+0000", 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.26e-06 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.599261+0000", 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000441097 2024-04-18T23:12:01.959 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.599283+0000", 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1954999999999999e-05 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.599293+0000", 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0183999999999999e-05 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.599394+0000", 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010093 2024-04-18T23:12:01.960 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.599400+0000", 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1959999999999999e-06 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.640919+0000", 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.041519751000000001 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.640997+0000", 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7904999999999994e-05 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.961 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.641828+0000", 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000831246 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.644367+0000", 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.002538935 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=151, last_complete=95'111, committed=1, applied=1))", 2024-04-18T23:12:01.962 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.847451+0000", 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.088021846000000001, 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.088442457000000002, 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847451+0000", 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847451+0000", 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.963 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847452+0000", 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4900000000000005e-07 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847467+0000", 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4564e-05 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847480+0000", 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3236e-05 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.964 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847488+0000", 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7230000000000004e-06 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:838 3.4s0 3:2f7b52be:::smithi14230322-18:head [append 0~475136 in=475136b] snapc 22=[22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.602783+0000", 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.33269055400000003, 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.33315296900000002, 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.965 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 838 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.602783+0000", 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.602783+0000", 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.966 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.602784+0000", 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.782e-06 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603270+0000", 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00048526500000000002 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603290+0000", 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.016e-05 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.967 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603302+0000", 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1725999999999999e-05 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603390+0000", 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.8593999999999999e-05 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603399+0000", 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9090000000000006e-06 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.642428+0000", 2024-04-18T23:12:01.968 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.039029411 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.642549+0000", 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000120222 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.643549+0000", 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010006990000000001 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.744678+0000", 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101128879 2024-04-18T23:12:01.969 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=152, last_complete=95'112, committed=1, applied=1))", 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.847928+0000", 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.087545031999999995, 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.088076341000000002, 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.970 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847928+0000", 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847928+0000", 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847929+0000", 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.5799999999999998e-07 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.971 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847935+0000", 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6590000000000004e-06 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847949+0000", 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4221e-05 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.847960+0000", 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0645999999999999e-05 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.972 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:839 3.4s0 3:2f7b52be:::smithi14230322-18:head [append 0~98304 in=98304b] snapc 22=[22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.603536+0000", 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.33193759, 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.33251059500000002, 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 839 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.973 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603536+0000", 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603536+0000", 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603536+0000", 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8800000000000002e-07 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.974 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603662+0000", 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000126043 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603672+0000", 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.9380000000000004e-06 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603683+0000", 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.093e-05 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603762+0000", 2024-04-18T23:12:01.975 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.8548999999999996e-05 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603770+0000", 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4819999999999994e-06 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.744993+0000", 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.141222716 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.745131+0000", 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000138417 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.976 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.745837+0000", 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00070608800000000001 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.747951+0000", 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0021131909999999999 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=147, last_complete=95'107, committed=1, applied=1))", 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.848499+0000", 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.086974300000000004, 2024-04-18T23:12:01.977 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.087616816, 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848499+0000", 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848499+0000", 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.978 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848500+0000", 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3600000000000002e-07 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848503+0000", 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5109999999999999e-06 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848511+0000", 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4329999999999998e-06 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.979 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848519+0000", 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2730000000000002e-06 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:840 3.4s0 3:2f7b52be:::smithi14230322-18:head [setxattr _header (57) in=64b] snapc 22=[22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e95)", 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.603703+0000", 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.33177012299999997, 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.33245567799999998, 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:01.980 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 840 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603703+0000", 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603703+0000", 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.981 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603704+0000", 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9599999999999999e-07 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603710+0000", 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6100000000000002e-06 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603717+0000", 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5359999999999998e-06 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.982 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603723+0000", 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3810000000000004e-06 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603803+0000", 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0084000000000001e-05 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603812+0000", 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3720000000000005e-06 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.983 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.746413+0000", 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.14260178900000001 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.746482+0000", 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9000999999999999e-05 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.746815+0000", 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00033211900000000002 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.748256+0000", 2024-04-18T23:12:01.984 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001441026 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=148, last_complete=95'108, committed=1, applied=1))", 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.848536+0000", 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.086937400999999997, 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.087692052000000006, 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848536+0000", 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.985 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848536+0000", 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848536+0000", 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8000000000000001e-07 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848539+0000", 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7489999999999999e-06 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.986 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848548+0000", 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.054e-06 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848555+0000", 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6769999999999998e-06 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:841 3.4s0 3:2f7b52be:::smithi14230322-18:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e95)", 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.603745+0000", 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.33172795399999999, 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.33252821599999999, 2024-04-18T23:12:01.987 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "waiting for rw locks", 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 841 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603745+0000", 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603745+0000", 2024-04-18T23:12:01.988 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603746+0000", 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9800000000000003e-07 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603748+0000", 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6120000000000001e-06 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.989 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603761+0000", 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2121e-05 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603767+0000", 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8140000000000003e-06 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603841+0000", 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3909e-05 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.990 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.603850+0000", 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7499999999999992e-06 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.747052+0000", 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.14320179399999999 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.747156+0000", 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104252 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.991 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=149, last_complete=95'109, committed=1, applied=1))", 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.848565+0000", 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.086908535999999995, 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.087769101000000002, 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848565+0000", 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.992 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848565+0000", 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848565+0000", 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2599999999999998e-07 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848568+0000", 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7130000000000001e-06 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848575+0000", 2024-04-18T23:12:01.993 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6079999999999998e-06 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848582+0000", 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4280000000000001e-06 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=150, last_complete=95'110, committed=1, applied=1))", 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.848590+0000", 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.086882754000000006, 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.087780559999999994, 2024-04-18T23:12:01.994 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848590+0000", 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848590+0000", 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.995 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848591+0000", 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8799999999999998e-07 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848594+0000", 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8880000000000001e-06 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848603+0000", 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.1209999999999996e-06 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848611+0000", 2024-04-18T23:12:01.996 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9910000000000002e-06 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=151, last_complete=95'111, committed=1, applied=1))", 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.848620+0000", 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.086853538999999993, 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.087787744000000001, 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.997 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848620+0000", 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848620+0000", 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848620+0000", 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8500000000000002e-07 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:01.998 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848623+0000", 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.65e-06 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848632+0000", 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3149999999999998e-06 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848638+0000", 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3470000000000004e-06 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:01.999 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=152, last_complete=95'112, committed=1, applied=1))", 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.848647+0000", 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.086825992000000005, 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.087796921, 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848647+0000", 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.000 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848647+0000", 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848647+0000", 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.9700000000000003e-07 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848650+0000", 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4839999999999998e-06 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.001 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848658+0000", 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3699999999999995e-06 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848664+0000", 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0739999999999998e-06 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.4s0 95/31 ECSubWriteReply(tid=151, last_complete=95'111, committed=1, applied=1))", 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.848968+0000", 2024-04-18T23:12:02.002 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 0.086505545000000003, 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.087513366999999995, 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848968+0000", 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848968+0000", 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:02.003 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848969+0000", 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.077e-06 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848977+0000", 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0930000000000003e-06 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.848997+0000", 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9925999999999999e-05 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:02.004 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.849007+0000", 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0547000000000001e-05 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout: ], 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 20 2024-04-18T23:12:02.005 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:02.006 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:12:02.006 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:12:02.007 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:12:02.008 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:12:02.008 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:12:02.008 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:12:02.008 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:12:02.009 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:02.002+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:02.014 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:12:02.014 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:12:02.035 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:12:02.035 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:12:02.047 INFO:tasks.rados.rados.0.smithi142.stdout:337: finishing write tid 1 to smithi14230322-18 2024-04-18T23:12:02.062 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:12:02.063 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:12:02.082 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:12:02.083 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:12:02.106 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:12:02.107 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:12:02.107 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:02.106+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:02.114 INFO:tasks.rados.rados.0.smithi142.stdout:338: expect (ObjNum 115 snap 26 seq_num 115) 2024-04-18T23:12:02.126 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:12:02.127 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:12:02.139 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:02.139 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:02.150 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:12:02.150 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:12:02.153 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:12:02.172 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:12:02.172 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:12:02.195 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:12:02.195 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:12:02.208 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:02.206+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:02.247 INFO:tasks.rados.rados.0.smithi142.stdout:337: finishing write tid 2 to smithi14230322-18 2024-04-18T23:12:02.247 INFO:tasks.rados.rados.0.smithi142.stdout:337: finishing write tid 3 to smithi14230322-18 2024-04-18T23:12:02.291 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:02.292 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:02.306 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:12:02.309 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:12:02.308+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:12:02.348 INFO:tasks.rados.rados.0.smithi142.stdout:337: finishing write tid 4 to smithi14230322-18 2024-04-18T23:12:02.348 INFO:tasks.rados.rados.0.smithi142.stdout:337: finishing write tid 5 to smithi14230322-18 2024-04-18T23:12:02.410 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:12:02.411+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:12:02.429 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:02.429 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:02.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:12:02.448 INFO:tasks.rados.rados.0.smithi142.stdout:337: finishing write tid 6 to smithi14230322-18 2024-04-18T23:12:02.511 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:12:02.508+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:12:02.544 INFO:tasks.rados.rados.0.smithi142.stdout:338: done (4 left) 2024-04-18T23:12:02.544 INFO:tasks.rados.rados.0.smithi142.stdout:339: done (3 left) 2024-04-18T23:12:02.544 INFO:tasks.rados.rados.0.smithi142.stdout:340: done (2 left) 2024-04-18T23:12:02.544 INFO:tasks.rados.rados.0.smithi142.stdout:341: done (1 left) 2024-04-18T23:12:02.544 INFO:tasks.rados.rados.0.smithi142.stdout:342: read oid 43 snap -1 2024-04-18T23:12:02.544 INFO:tasks.rados.rados.0.smithi142.stdout:342: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-18T23:12:02.545 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 47 current snap is 34 2024-04-18T23:12:02.545 INFO:tasks.rados.rados.0.smithi142.stdout:343: seq_num 126 ranges {786432=573440,1359872=622592,1982464=622592,2605056=311296} 2024-04-18T23:12:02.551 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T23:12:02.554 INFO:tasks.rados.rados.0.smithi142.stdout:343: writing smithi14230322-47 from 786432 to 1359872 tid 1 2024-04-18T23:12:02.559 INFO:tasks.rados.rados.0.smithi142.stdout:343: writing smithi14230322-47 from 1359872 to 1982464 tid 2 2024-04-18T23:12:02.563 INFO:tasks.rados.rados.0.smithi142.stdout:343: writing smithi14230322-47 from 1982464 to 2605056 tid 3 2024-04-18T23:12:02.566 INFO:tasks.rados.rados.0.smithi142.stdout:343: writing smithi14230322-47 from 2605056 to 2916352 tid 4 2024-04-18T23:12:02.566 INFO:tasks.rados.rados.0.smithi142.stdout:344: setattr oid 40 current snap is 34 2024-04-18T23:12:02.567 INFO:tasks.rados.rados.0.smithi142.stdout:345: delete oid 34 current snap is 34 2024-04-18T23:12:02.571 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:02.571 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:02.587 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:12:02.611 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:02.612+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:02.652 INFO:tasks.rados.rados.0.smithi142.stdout:337: finishing write tid 7 to smithi14230322-18 2024-04-18T23:12:02.652 INFO:tasks.rados.rados.0.smithi142.stdout:337: oid 18 updating version 0 to 110 2024-04-18T23:12:02.652 INFO:tasks.rados.rados.0.smithi142.stdout:337: oid 18 updating version 110 to 112 2024-04-18T23:12:02.652 INFO:tasks.rados.rados.0.smithi142.stdout:337: oid 18 version 112 is already newer than 108 2024-04-18T23:12:02.652 INFO:tasks.rados.rados.0.smithi142.stdout:337: oid 18 version 112 is already newer than 109 2024-04-18T23:12:02.652 INFO:tasks.rados.rados.0.smithi142.stdout:337: oid 18 version 112 is already newer than 107 2024-04-18T23:12:02.652 INFO:tasks.rados.rados.0.smithi142.stdout:337: oid 18 version 112 is already newer than 111 2024-04-18T23:12:02.652 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 18 v 112 (ObjNum 125 snap 33 seq_num 125) dirty exists 2024-04-18T23:12:02.652 INFO:tasks.rados.rados.0.smithi142.stdout:337: left oid 18 (ObjNum 125 snap 33 seq_num 125) 2024-04-18T23:12:02.712 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:02.711+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:02.755 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:02.755 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:02.772 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:12:02.776 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 40 v 126 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-18T23:12:02.812 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:12:02.812+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:12:02.903 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:02.903 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:02.913 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:02.912+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:02.918 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:12:02.964 INFO:tasks.rados.rados.0.smithi142.stdout:342: expect (ObjNum 72 snap 11 seq_num 72) 2024-04-18T23:12:03.014 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:03.012+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:03.036 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:03.037 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:03.051 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:12:03.080 INFO:tasks.rados.rados.0.smithi142.stdout:343: finishing write tid 1 to smithi14230322-47 2024-04-18T23:12:03.080 INFO:tasks.rados.rados.0.smithi142.stdout:337: done (4 left) 2024-04-18T23:12:03.080 INFO:tasks.rados.rados.0.smithi142.stdout:342: done (3 left) 2024-04-18T23:12:03.080 INFO:tasks.rados.rados.0.smithi142.stdout:344: done (2 left) 2024-04-18T23:12:03.080 INFO:tasks.rados.rados.0.smithi142.stdout:345: done (1 left) 2024-04-18T23:12:03.080 INFO:tasks.rados.rados.0.smithi142.stdout:346: read oid 20 snap -1 2024-04-18T23:12:03.080 INFO:tasks.rados.rados.0.smithi142.stdout:346: expect (ObjNum 95 snap 21 seq_num 95) 2024-04-18T23:12:03.081 INFO:tasks.rados.rados.0.smithi142.stdout:347: copy_from oid 40 from oid 27 current snap is 34 2024-04-18T23:12:03.081 INFO:tasks.rados.rados.0.smithi142.stdout:343: finishing write tid 2 to smithi14230322-47 2024-04-18T23:12:03.081 INFO:tasks.rados.rados.0.smithi142.stdout:343: finishing write tid 3 to smithi14230322-47 2024-04-18T23:12:03.081 INFO:tasks.rados.rados.0.smithi142.stdout:343: finishing write tid 4 to smithi14230322-47 2024-04-18T23:12:03.081 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 38 current snap is 34 2024-04-18T23:12:03.081 INFO:tasks.rados.rados.0.smithi142.stdout:348: seq_num 128 ranges {1949696=606208,2555904=507904,3063808=573440,3637248=475136} 2024-04-18T23:12:03.102 INFO:tasks.rados.rados.0.smithi142.stdout:348: writing smithi14230322-38 from 1949696 to 2555904 tid 1 2024-04-18T23:12:03.106 INFO:tasks.rados.rados.0.smithi142.stdout:348: writing smithi14230322-38 from 2555904 to 3063808 tid 2 2024-04-18T23:12:03.111 INFO:tasks.rados.rados.0.smithi142.stdout:348: writing smithi14230322-38 from 3063808 to 3637248 tid 3 2024-04-18T23:12:03.114 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:03.110+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:03.115 INFO:tasks.rados.rados.0.smithi142.stdout:348: writing smithi14230322-38 from 3637248 to 4112384 tid 4 2024-04-18T23:12:03.116 INFO:tasks.rados.rados.0.smithi142.stdout:349: read oid 41 snap -1 2024-04-18T23:12:03.116 INFO:tasks.rados.rados.0.smithi142.stdout:349: expect deleted 2024-04-18T23:12:03.116 INFO:tasks.rados.rados.0.smithi142.stdout:350: rollback oid 42 current snap is 34 2024-04-18T23:12:03.116 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 42 to 31 2024-04-18T23:12:03.116 INFO:tasks.rados.rados.0.smithi142.stdout:343: finishing write tid 5 to smithi14230322-47 2024-04-18T23:12:03.116 INFO:tasks.rados.rados.0.smithi142.stdout:351: read oid 30 snap -1 2024-04-18T23:12:03.116 INFO:tasks.rados.rados.0.smithi142.stdout:351: expect (ObjNum 113 snap 26 seq_num 113) 2024-04-18T23:12:03.116 INFO:tasks.rados.rados.0.smithi142.stdout:352: delete oid 50 current snap is 34 2024-04-18T23:12:03.175 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:03.175 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:03.189 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:12:03.215 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:12:03.215+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:12:03.316 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:12:03.315+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:12:03.324 INFO:tasks.rados.rados.0.smithi142.stdout:350: finishing rollback tid 0 to smithi14230322-42 2024-04-18T23:12:03.326 INFO:tasks.rados.rados.0.smithi142.stdout:350: finishing rollback tid 1 to smithi14230322-42 2024-04-18T23:12:03.326 INFO:tasks.rados.rados.0.smithi142.stdout:350: finishing rollback tid 2 to smithi14230322-42 2024-04-18T23:12:03.326 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 42 v 130 (ObjNum 109 snap 25 seq_num 109) dirty exists 2024-04-18T23:12:03.333 INFO:tasks.rados.rados.0.smithi142.stdout:347: finishing copy_from racing read to smithi14230322-40 2024-04-18T23:12:03.357 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:03.358 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:03.371 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:12:03.397 INFO:tasks.rados.rados.0.smithi142.stdout:346: expect (ObjNum 95 snap 21 seq_num 95) 2024-04-18T23:12:03.419 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:03.418+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:03.490 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:03.490 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:03.503 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:12:03.524 INFO:tasks.rados.rados.0.smithi142.stdout:343: finishing write tid 6 to smithi14230322-47 2024-04-18T23:12:03.524 INFO:tasks.rados.rados.0.smithi142.stdout:343: oid 47 updating version 0 to 211 2024-04-18T23:12:03.524 INFO:tasks.rados.rados.0.smithi142.stdout:343: oid 47 version 211 is already newer than 209 2024-04-18T23:12:03.524 INFO:tasks.rados.rados.0.smithi142.stdout:343: oid 47 updating version 211 to 212 2024-04-18T23:12:03.524 INFO:tasks.rados.rados.0.smithi142.stdout:343: oid 47 version 212 is already newer than 210 2024-04-18T23:12:03.524 INFO:tasks.rados.rados.0.smithi142.stdout:343: oid 47 version 212 is already newer than 208 2024-04-18T23:12:03.524 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 47 v 212 (ObjNum 126 snap 34 seq_num 126) dirty exists 2024-04-18T23:12:03.524 INFO:tasks.rados.rados.0.smithi142.stdout:343: left oid 47 (ObjNum 126 snap 34 seq_num 126) 2024-04-18T23:12:03.524 INFO:tasks.rados.rados.0.smithi142.stdout:351: expect (ObjNum 113 snap 26 seq_num 113) 2024-04-18T23:12:03.547 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:03.546+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:03.558 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-04-18T23:12:03.591 INFO:tasks.rados.rados.0.smithi142.stdout:347: finishing copy_from to smithi14230322-40 2024-04-18T23:12:03.592 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 40 v 33 (ObjNum 26 snap 0 seq_num 26) dirty exists 2024-04-18T23:12:03.619 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:03.620+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:03.664 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:03.664 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:03.678 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:12:03.720 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:12:03.718+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:12:03.800 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:03.800 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:03.811 INFO:tasks.rados.rados.0.smithi142.stdout:348: finishing write tid 1 to smithi14230322-38 2024-04-18T23:12:03.812 INFO:tasks.rados.rados.0.smithi142.stdout:348: finishing write tid 2 to smithi14230322-38 2024-04-18T23:12:03.812 INFO:tasks.rados.rados.0.smithi142.stdout:348: finishing write tid 3 to smithi14230322-38 2024-04-18T23:12:03.816 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:12:03.821 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:03.820+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:03.912 INFO:tasks.rados.rados.0.smithi142.stdout:348: finishing write tid 4 to smithi14230322-38 2024-04-18T23:12:03.922 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:03.923+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:03.939 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:03.939 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:03.955 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:12:03.970 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-18T23:12:03.970 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-18T23:12:03.970 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:12:04.013 INFO:tasks.rados.rados.0.smithi142.stdout:348: finishing write tid 5 to smithi14230322-38 2024-04-18T23:12:04.014 INFO:tasks.rados.rados.0.smithi142.stdout:343: done (7 left) 2024-04-18T23:12:04.014 INFO:tasks.rados.rados.0.smithi142.stdout:346: done (6 left) 2024-04-18T23:12:04.014 INFO:tasks.rados.rados.0.smithi142.stdout:347: done (5 left) 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:349: done (4 left) 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:350: done (3 left) 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:351: done (2 left) 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:352: done (1 left) 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:353: rollback oid 26 current snap is 34 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 26 to 27 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:354: rollback oid 37 current snap is 34 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 37 to 30 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 14 current snap is 34 2024-04-18T23:12:04.015 INFO:tasks.rados.rados.0.smithi142.stdout:355: seq_num 130 ranges {229376=638976,868352=671744,1540096=802816,2342912=81920} 2024-04-18T23:12:04.022 INFO:tasks.rados.rados.0.smithi142.stdout:355: writing smithi14230322-14 from 229376 to 868352 tid 1 2024-04-18T23:12:04.023 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:04.023+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:04.028 INFO:tasks.rados.rados.0.smithi142.stdout:355: writing smithi14230322-14 from 868352 to 1540096 tid 2 2024-04-18T23:12:04.036 INFO:tasks.rados.rados.0.smithi142.stdout:355: writing smithi14230322-14 from 1540096 to 2342912 tid 3 2024-04-18T23:12:04.037 INFO:tasks.rados.rados.0.smithi142.stdout:355: writing smithi14230322-14 from 2342912 to 2424832 tid 4 2024-04-18T23:12:04.037 INFO:tasks.rados.rados.0.smithi142.stdout:356: read oid 5 snap -1 2024-04-18T23:12:04.037 INFO:tasks.rados.rados.0.smithi142.stdout:356: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-18T23:12:04.037 INFO:tasks.rados.rados.0.smithi142.stdout:357: rollback oid 36 current snap is 34 2024-04-18T23:12:04.037 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 36 to 27 2024-04-18T23:12:04.037 INFO:tasks.rados.rados.0.smithi142.stdout:358: snap_create 2024-04-18T23:12:04.077 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:04.077 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:04.091 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:12:04.124 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:04.124+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:04.214 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:04.214 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:04.214 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:12:04.220 INFO:tasks.rados.rados.0.smithi142.stdout:348: finishing write tid 6 to smithi14230322-38 2024-04-18T23:12:04.220 INFO:tasks.rados.rados.0.smithi142.stdout:348: oid 38 updating version 0 to 217 2024-04-18T23:12:04.220 INFO:tasks.rados.rados.0.smithi142.stdout:348: oid 38 version 217 is already newer than 215 2024-04-18T23:12:04.220 INFO:tasks.rados.rados.0.smithi142.stdout:348: oid 38 updating version 217 to 218 2024-04-18T23:12:04.220 INFO:tasks.rados.rados.0.smithi142.stdout:348: oid 38 version 218 is already newer than 216 2024-04-18T23:12:04.220 INFO:tasks.rados.rados.0.smithi142.stdout:348: oid 38 version 218 is already newer than 214 2024-04-18T23:12:04.220 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 38 v 218 (ObjNum 128 snap 34 seq_num 128) dirty exists 2024-04-18T23:12:04.220 INFO:tasks.rados.rados.0.smithi142.stdout:348: left oid 38 (ObjNum 128 snap 34 seq_num 128) 2024-04-18T23:12:04.225 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:04.223+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:04.228 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:12:04.285 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:12:04.286 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:12:04.305 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":123,"stamp":"2024-04-18T23:12:03.554247+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":101761568,"num_objects":110,"num_object_clones":58,"num_object_copies":760,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":5,"num_read":546,"num_read_kb":121088,"num_write":950,"num_write_kb":122452,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1468,"ondisk_log_size":1468,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":678852,"kb_used_data":195036,"kb_used_omap":35,"kb_used_meta":483548,"kb_avail":1509270588,"statfs":{"total":1546188226560,"available":1545493082112,"internally_reserved":0,"allocated":199716864,"data_stored":268661617,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":495153793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,2,0,2],"upper_bound":256},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4177920,"num_objects":5,"num_object_clones":5,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":10,"num_read_kb":24240,"num_write":16,"num_write_kb":4080,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.042706"},"pg_stats":[{"pgid":"3.f","version":"80'19","reported_seq":317,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.580748+0000","last_change":"2024-04-18T23:12:00.581123+0000","last_active":"2024-04-18T23:12:00.580748+0000","last_peered":"2024-04-18T23:12:00.580748+0000","last_clean":"2024-04-18T23:12:00.580748+0000","last_became_active":"2024-04-18T23:10:54.696778+0000","last_became_peered":"2024-04-18T23:10:54.696778+0000","last_unstale":"2024-04-18T23:12:00.580748+0000","last_undegraded":"2024-04-18T23:12:00.580748+0000","last_fullsized":"2024-04-18T23:12:00.580748+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:18.058346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00060988100000000005,"stat_sum":{"num_bytes":1933312,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":15,"num_read_kb":13445,"num_write":6,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.e","version":"87'84","reported_seq":384,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:01.431567+0000","last_change":"2024-04-18T23:12:01.432288+0000","last_active":"2024-04-18T23:12:01.431567+0000","last_peered":"2024-04-18T23:12:01.431567+0000","last_clean":"2024-04-18T23:12:01.431567+0000","last_became_active":"2024-04-18T23:10:54.918517+0000","last_became_peered":"2024-04-18T23:10:54.918517+0000","last_unstale":"2024-04-18T23:12:01.431567+0000","last_undegraded":"2024-04-18T23:12:01.431567+0000","last_fullsized":"2024-04-18T23:12:01.431567+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'84","last_scrub_stamp":"2024-04-18T23:12:01.429717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.429717+0000","objects_scrubbed":6,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:10.507212+0000","scrub_duration":815,"objects_trimmed":0,"snaptrim_duration":0.71950589399999998,"stat_sum":{"num_bytes":6995968,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":16277,"num_write":54,"num_write_kb":8016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.d","version":"87'148","reported_seq":448,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-18T23:12:01.613170+0000","last_change":"2024-04-18T23:12:00.556457+0000","last_active":"2024-04-18T23:12:01.613170+0000","last_peered":"2024-04-18T23:12:01.613170+0000","last_clean":"2024-04-18T23:12:01.613170+0000","last_became_active":"2024-04-18T23:10:54.712663+0000","last_became_peered":"2024-04-18T23:10:54.712663+0000","last_unstale":"2024-04-18T23:12:01.613170+0000","last_undegraded":"2024-04-18T23:12:01.613170+0000","last_fullsized":"2024-04-18T23:12:01.613170+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001652335,"stat_sum":{"num_bytes":11730944,"num_objects":13,"num_object_clones":8,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":51,"num_read_kb":5297,"num_write":103,"num_write_kb":15088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.c","version":"87'61","reported_seq":354,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.580290+0000","last_change":"2024-04-18T23:12:00.580520+0000","last_active":"2024-04-18T23:12:00.580290+0000","last_peered":"2024-04-18T23:12:00.580290+0000","last_clean":"2024-04-18T23:12:00.580290+0000","last_became_active":"2024-04-18T23:10:54.717053+0000","last_became_peered":"2024-04-18T23:10:54.717053+0000","last_unstale":"2024-04-18T23:12:00.580290+0000","last_undegraded":"2024-04-18T23:12:00.580290+0000","last_fullsized":"2024-04-18T23:12:00.580290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:03.010529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035773199999999998,"stat_sum":{"num_bytes":6471680,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":17,"num_read_kb":1926,"num_write":35,"num_write_kb":6976,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.b","version":"94'175","reported_seq":467,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:01.474880+0000","last_change":"2024-04-18T23:12:01.277275+0000","last_active":"2024-04-18T23:12:01.474880+0000","last_peered":"2024-04-18T23:12:01.474880+0000","last_clean":"2024-04-18T23:12:01.474880+0000","last_became_active":"2024-04-18T23:10:54.808152+0000","last_became_peered":"2024-04-18T23:10:54.808152+0000","last_unstale":"2024-04-18T23:12:01.474880+0000","last_undegraded":"2024-04-18T23:12:01.474880+0000","last_fullsized":"2024-04-18T23:12:01.474880+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:16.208503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.72976707100000004,"stat_sum":{"num_bytes":17809408,"num_objects":14,"num_object_clones":9,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":1,"num_read":39,"num_read_kb":26755,"num_write":77,"num_write_kb":17664,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.a","version":"87'24","reported_seq":330,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.646211+0000","last_change":"2024-04-18T23:12:00.846769+0000","last_active":"2024-04-18T23:12:00.646211+0000","last_peered":"2024-04-18T23:12:00.646211+0000","last_clean":"2024-04-18T23:12:00.646211+0000","last_became_active":"2024-04-18T23:10:54.764999+0000","last_became_peered":"2024-04-18T23:10:54.764999+0000","last_unstale":"2024-04-18T23:12:00.646211+0000","last_undegraded":"2024-04-18T23:12:00.646211+0000","last_fullsized":"2024-04-18T23:12:00.646211+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:10.970988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.200709421,"stat_sum":{"num_bytes":720896,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":3750,"num_write":6,"num_write_kb":624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.9","version":"87'124","reported_seq":473,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.618839+0000","last_change":"2024-04-18T23:12:00.619403+0000","last_active":"2024-04-18T23:12:00.618839+0000","last_peered":"2024-04-18T23:12:00.618839+0000","last_clean":"2024-04-18T23:12:00.618839+0000","last_became_active":"2024-04-18T23:11:39.726824+0000","last_became_peered":"2024-04-18T23:11:39.726824+0000","last_unstale":"2024-04-18T23:12:00.618839+0000","last_undegraded":"2024-04-18T23:12:00.618839+0000","last_fullsized":"2024-04-18T23:12:00.618839+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'124","last_scrub_stamp":"2024-04-18T23:11:59.890703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:11:59.890703+0000","objects_scrubbed":13,"log_size":124,"log_dups_size":0,"ondisk_log_size":124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:10.071037+0000","scrub_duration":619,"objects_trimmed":0,"snaptrim_duration":0.00090971900000000002,"stat_sum":{"num_bytes":5324800,"num_objects":13,"num_object_clones":7,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":55,"num_read_kb":10769,"num_write":81,"num_write_kb":7696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2867200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.8","version":"87'61","reported_seq":385,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.875097+0000","last_change":"2024-04-18T23:12:00.875929+0000","last_active":"2024-04-18T23:12:00.875097+0000","last_peered":"2024-04-18T23:12:00.875097+0000","last_clean":"2024-04-18T23:12:00.875097+0000","last_became_active":"2024-04-18T23:11:37.876250+0000","last_became_peered":"2024-04-18T23:11:37.876250+0000","last_unstale":"2024-04-18T23:12:00.875097+0000","last_undegraded":"2024-04-18T23:12:00.875097+0000","last_fullsized":"2024-04-18T23:12:00.875097+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001221682,"stat_sum":{"num_bytes":1949696,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":8055,"num_write":54,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"1.0","version":"33'157","reported_seq":356,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.618545+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:12:00.618545+0000","last_peered":"2024-04-18T23:12:00.618545+0000","last_clean":"2024-04-18T23:12:00.618545+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:12:00.618545+0000","last_undegraded":"2024-04-18T23:12:00.618545+0000","last_fullsized":"2024-04-18T23:12:00.618545+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.874528+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:12:00.874528+0000","last_peered":"2024-04-18T23:12:00.874528+0000","last_clean":"2024-04-18T23:12:00.874528+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:12:00.874528+0000","last_undegraded":"2024-04-18T23:12:00.874528+0000","last_fullsized":"2024-04-18T23:12:00.874528+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"94'85","reported_seq":404,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:01.123763+0000","last_change":"2024-04-18T23:12:01.125202+0000","last_active":"2024-04-18T23:12:01.123763+0000","last_peered":"2024-04-18T23:12:01.123763+0000","last_clean":"2024-04-18T23:12:01.123763+0000","last_became_active":"2024-04-18T23:11:33.818951+0000","last_became_peered":"2024-04-18T23:11:33.818951+0000","last_unstale":"2024-04-18T23:12:01.123763+0000","last_undegraded":"2024-04-18T23:12:01.123763+0000","last_fullsized":"2024-04-18T23:12:01.123763+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":85,"log_dups_size":0,"ondisk_log_size":85,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:13.336217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.40641092299999998,"stat_sum":{"num_bytes":9797632,"num_objects":8,"num_object_clones":4,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":26,"num_read_kb":6012,"num_write":38,"num_write_kb":9872,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"2.0","version":"0'0","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:01.049827+0000","last_change":"2024-04-18T23:12:01.049893+0000","last_active":"2024-04-18T23:12:01.049827+0000","last_peered":"2024-04-18T23:12:01.049827+0000","last_clean":"2024-04-18T23:12:01.049827+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:12:01.049827+0000","last_undegraded":"2024-04-18T23:12:01.049827+0000","last_fullsized":"2024-04-18T23:12:01.049827+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:01.049599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.049599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:09.776200+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"88'10","reported_seq":304,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.547488+0000","last_change":"2024-04-18T23:12:00.548488+0000","last_active":"2024-04-18T23:12:00.547488+0000","last_peered":"2024-04-18T23:12:00.547488+0000","last_clean":"2024-04-18T23:12:00.547488+0000","last_became_active":"2024-04-18T23:10:54.799189+0000","last_became_peered":"2024-04-18T23:10:54.799189+0000","last_unstale":"2024-04-18T23:12:00.547488+0000","last_undegraded":"2024-04-18T23:12:00.547488+0000","last_fullsized":"2024-04-18T23:12:00.547488+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.453435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001356409,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"2.1","version":"0'0","reported_seq":146,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.618559+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:12:00.618559+0000","last_peered":"2024-04-18T23:12:00.618559+0000","last_clean":"2024-04-18T23:12:00.618559+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:12:00.618559+0000","last_undegraded":"2024-04-18T23:12:00.618559+0000","last_fullsized":"2024-04-18T23:12:00.618559+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:14.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"87'44","reported_seq":370,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.518898+0000","last_change":"2024-04-18T23:12:00.519578+0000","last_active":"2024-04-18T23:12:00.518898+0000","last_peered":"2024-04-18T23:12:00.518898+0000","last_clean":"2024-04-18T23:12:00.518898+0000","last_became_active":"2024-04-18T23:11:38.818022+0000","last_became_peered":"2024-04-18T23:11:38.818022+0000","last_unstale":"2024-04-18T23:12:00.518898+0000","last_undegraded":"2024-04-18T23:12:00.518898+0000","last_fullsized":"2024-04-18T23:12:00.518898+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099991500000000009,"stat_sum":{"num_bytes":6438912,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":2792,"num_write":20,"num_write_kb":6288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":7716864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"2.2","version":"0'0","reported_seq":145,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.746257+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:12:00.746257+0000","last_peered":"2024-04-18T23:12:00.746257+0000","last_clean":"2024-04-18T23:12:00.746257+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:12:00.746257+0000","last_undegraded":"2024-04-18T23:12:00.746257+0000","last_fullsized":"2024-04-18T23:12:00.746257+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.994274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":306,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.619371+0000","last_change":"2024-04-18T23:12:00.620110+0000","last_active":"2024-04-18T23:12:00.619371+0000","last_peered":"2024-04-18T23:12:00.619371+0000","last_clean":"2024-04-18T23:12:00.619371+0000","last_became_active":"2024-04-18T23:11:33.713370+0000","last_became_peered":"2024-04-18T23:11:33.713370+0000","last_unstale":"2024-04-18T23:12:00.619371+0000","last_undegraded":"2024-04-18T23:12:00.619371+0000","last_fullsized":"2024-04-18T23:12:00.619371+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001258564,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.5","version":"94'123","reported_seq":440,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-18T23:12:01.910249+0000","last_change":"2024-04-18T23:12:00.864805+0000","last_active":"2024-04-18T23:12:01.910249+0000","last_peered":"2024-04-18T23:12:01.910249+0000","last_clean":"2024-04-18T23:12:01.910249+0000","last_became_active":"2024-04-18T23:11:39.923911+0000","last_became_peered":"2024-04-18T23:11:39.923911+0000","last_unstale":"2024-04-18T23:12:01.910249+0000","last_undegraded":"2024-04-18T23:12:01.910249+0000","last_fullsized":"2024-04-18T23:12:01.910249+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.244857663,"stat_sum":{"num_bytes":6455296,"num_objects":9,"num_object_clones":6,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":40,"num_read_kb":12302,"num_write":59,"num_write_kb":11760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5685248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"2.4","version":"0'0","reported_seq":154,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.547004+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:12:00.547004+0000","last_peered":"2024-04-18T23:12:00.547004+0000","last_clean":"2024-04-18T23:12:00.547004+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:12:00.547004+0000","last_undegraded":"2024-04-18T23:12:00.547004+0000","last_fullsized":"2024-04-18T23:12:00.547004+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"88'105","reported_seq":388,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.680940+0000","last_change":"2024-04-18T23:12:00.681775+0000","last_active":"2024-04-18T23:12:00.680940+0000","last_peered":"2024-04-18T23:12:00.680940+0000","last_clean":"2024-04-18T23:12:00.680940+0000","last_became_active":"2024-04-18T23:10:54.793319+0000","last_became_peered":"2024-04-18T23:10:54.793319+0000","last_unstale":"2024-04-18T23:12:00.680940+0000","last_undegraded":"2024-04-18T23:12:00.680940+0000","last_fullsized":"2024-04-18T23:12:00.680940+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:18.643918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001194676,"stat_sum":{"num_bytes":9486336,"num_objects":9,"num_object_clones":5,"num_object_copies":63,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":16,"num_read_kb":7,"num_write":29,"num_write_kb":9968,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"2.5","version":"0'0","reported_seq":146,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.555943+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:12:00.555943+0000","last_peered":"2024-04-18T23:12:00.555943+0000","last_clean":"2024-04-18T23:12:00.555943+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:12:00.555943+0000","last_undegraded":"2024-04-18T23:12:00.555943+0000","last_fullsized":"2024-04-18T23:12:00.555943+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.503236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"88'206","reported_seq":493,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.548584+0000","last_change":"2024-04-18T23:12:00.548987+0000","last_active":"2024-04-18T23:12:00.548584+0000","last_peered":"2024-04-18T23:12:00.548584+0000","last_clean":"2024-04-18T23:12:00.548584+0000","last_became_active":"2024-04-18T23:10:54.693609+0000","last_became_peered":"2024-04-18T23:10:54.693609+0000","last_unstale":"2024-04-18T23:12:00.548584+0000","last_undegraded":"2024-04-18T23:12:00.548584+0000","last_fullsized":"2024-04-18T23:12:00.548584+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":206,"log_dups_size":0,"ondisk_log_size":206,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:17.926955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388440000000001,"stat_sum":{"num_bytes":12664832,"num_objects":14,"num_object_clones":6,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":48,"num_read_kb":11703,"num_write":163,"num_write_kb":14880,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":145,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.622532+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:12:00.622532+0000","last_peered":"2024-04-18T23:12:00.622532+0000","last_clean":"2024-04-18T23:12:00.622532+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:12:00.622532+0000","last_undegraded":"2024-04-18T23:12:00.622532+0000","last_fullsized":"2024-04-18T23:12:00.622532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:05.869947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"87'42","reported_seq":369,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-18T23:12:00.555703+0000","last_change":"2024-04-18T23:12:00.556263+0000","last_active":"2024-04-18T23:12:00.555703+0000","last_peered":"2024-04-18T23:12:00.555703+0000","last_clean":"2024-04-18T23:12:00.555703+0000","last_became_active":"2024-04-18T23:11:39.717443+0000","last_became_peered":"2024-04-18T23:11:39.717443+0000","last_unstale":"2024-04-18T23:12:00.555703+0000","last_undegraded":"2024-04-18T23:12:00.555703+0000","last_fullsized":"2024-04-18T23:12:00.555703+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:02.516171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012167580000000001,"stat_sum":{"num_bytes":3358720,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":17,"num_read_kb":1879,"num_write":18,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"2.7","version":"0'0","reported_seq":74,"reported_epoch":95,"state":"active+clean","last_fresh":"2024-04-18T23:12:01.810102+0000","last_change":"2024-04-18T23:11:33.519671+0000","last_active":"2024-04-18T23:12:01.810102+0000","last_peered":"2024-04-18T23:12:01.810102+0000","last_clean":"2024-04-18T23:12:01.810102+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:12:01.810102+0000","last_undegraded":"2024-04-18T23:12:01.810102+0000","last_fullsized":"2024-04-18T23:12:01.810102+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":101171200,"num_objects":108,"num_object_clones":58,"num_object_copies":756,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":5,"num_read":410,"num_read_kb":120970,"num_write":747,"num_write_kb":118992,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":177872896,"data_stored":260960256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0},"log_size":1311,"ondisk_log_size":1311,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808839,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38124,"kb_used_data":9572,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94333716,"statfs":{"total":96636764160,"available":96597725184,"internally_reserved":0,"allocated":9801728,"data_stored":13352100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280532,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41692,"kb_used_data":10504,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330148,"statfs":{"total":96636764160,"available":96594071552,"internally_reserved":0,"allocated":10756096,"data_stored":12242084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280532,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40636,"kb_used_data":9452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331204,"statfs":{"total":96636764160,"available":96595152896,"internally_reserved":0,"allocated":9678848,"data_stored":11894468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313235,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51376,"kb_used_data":20192,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320464,"statfs":{"total":96636764160,"available":96584155136,"internally_reserved":0,"allocated":20676608,"data_stored":27806884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47416,"kb_used_data":16232,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324424,"statfs":{"total":96636764160,"available":96588210176,"internally_reserved":0,"allocated":16621568,"data_stored":23686308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313236,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35064,"kb_used_data":8044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336776,"statfs":{"total":96636764160,"available":96600858624,"internally_reserved":0,"allocated":8237056,"data_stored":9872393,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313236,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48308,"kb_used_data":17124,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323532,"statfs":{"total":96636764160,"available":96587296768,"internally_reserved":0,"allocated":17534976,"data_stored":24448164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,2],"upper_bound":64},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313236,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35080,"kb_used_data":8060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94336760,"statfs":{"total":96636764160,"available":96600842240,"internally_reserved":0,"allocated":8253440,"data_stored":10028041,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313234,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47440,"kb_used_data":16256,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324400,"statfs":{"total":96636764160,"available":96588185600,"internally_reserved":0,"allocated":16646144,"data_stored":21605540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313234,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37760,"kb_used_data":10740,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334080,"statfs":{"total":96636764160,"available":96598097920,"internally_reserved":0,"allocated":10997760,"data_stored":14722057,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313235,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40464,"kb_used_data":9276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94331376,"statfs":{"total":96636764160,"available":96595329024,"internally_reserved":0,"allocated":9498624,"data_stored":13833225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,2,0,0,0,1],"upper_bound":256},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280532,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41976,"kb_used_data":10796,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329864,"statfs":{"total":96636764160,"available":96593780736,"internally_reserved":0,"allocated":11055104,"data_stored":14534185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280532,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48900,"kb_used_data":17720,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322940,"statfs":{"total":96636764160,"available":96586690560,"internally_reserved":0,"allocated":18145280,"data_stored":26858505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280533,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42316,"kb_used_data":11128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329524,"statfs":{"total":96636764160,"available":96593432576,"internally_reserved":0,"allocated":11395072,"data_stored":16651477,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280532,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42676,"kb_used_data":11496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329164,"statfs":{"total":96636764160,"available":96593063936,"internally_reserved":0,"allocated":11771904,"data_stored":17405141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280532,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39624,"kb_used_data":8444,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94332216,"statfs":{"total":96636764160,"available":96596189184,"internally_reserved":0,"allocated":8646656,"data_stored":9721045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":19296256,"data_stored":27746304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":15450112,"data_stored":23834624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":9158656,"data_stored":13209600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":6873088,"data_stored":9076736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16154624,"data_stored":23642112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6889472,"data_stored":9232384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":17305600,"data_stored":28696576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":9998336,"data_stored":15835136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":15265792,"data_stored":20799488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":10375168,"data_stored":16588800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":7249920,"data_stored":8904704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":9633792,"data_stored":13926400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":8658944,"data_stored":14925824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":7766016,"data_stored":10559488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":9375744,"data_stored":11436032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":8421376,"data_stored":12546048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0}]}} 2024-04-18T23:12:04.325 INFO:tasks.rados.rados.0.smithi142.stdout:353: finishing rollback tid 0 to smithi14230322-26 2024-04-18T23:12:04.326 INFO:tasks.rados.rados.0.smithi142.stdout:353: finishing rollback tid 1 to smithi14230322-26 2024-04-18T23:12:04.326 INFO:tasks.rados.rados.0.smithi142.stdout:353: finishing rollback tid 2 to smithi14230322-26 2024-04-18T23:12:04.326 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 26 v 135 (ObjNum 0 snap 0 seq_num 304) dirty exists 2024-04-18T23:12:04.326 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:04.324+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:04.348 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:04.348 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:04.360 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:12:04.360 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:12:04.363 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:12:04.378 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:12:04.378 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:12:04.427 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:04.424+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:04.445 INFO:tasks.rados.rados.0.smithi142.stdout:356: expect (ObjNum 4 snap 0 seq_num 4) 2024-04-18T23:12:04.448 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:12:04.449 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:12:04.475 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:12:04.475 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:12:04.487 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:04.487 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:04.500 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:12:04.500 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:12:04.502 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:12:04.524 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:12:04.524 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:12:04.527 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:04.526+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:04.545 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:12:04.545 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:12:04.558 INFO:tasks.rados.rados.0.smithi142.stdout:348: done (6 left) 2024-04-18T23:12:04.558 INFO:tasks.rados.rados.0.smithi142.stdout:353: done (5 left) 2024-04-18T23:12:04.558 INFO:tasks.rados.rados.0.smithi142.stdout:356: done (4 left) 2024-04-18T23:12:04.558 INFO:tasks.rados.rados.0.smithi142.stdout:358: done (3 left) 2024-04-18T23:12:04.558 INFO:tasks.rados.rados.0.smithi142.stdout:359: rollback oid 42 current snap is 35 2024-04-18T23:12:04.558 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 42 to 26 2024-04-18T23:12:04.558 INFO:tasks.rados.rados.0.smithi142.stdout:360: rmattr oid 20 current snap is 35 2024-04-18T23:12:04.558 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 48 current snap is 35 2024-04-18T23:12:04.558 INFO:tasks.rados.rados.0.smithi142.stdout:361: seq_num 131 ranges {491520=655360,1146880=425984,1572864=638976,2211840=491520} 2024-04-18T23:12:04.570 INFO:tasks.rados.rados.0.smithi142.stdout:361: writing smithi14230322-48 from 491520 to 1146880 tid 1 2024-04-18T23:12:04.571 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:12:04.571 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:12:04.574 INFO:tasks.rados.rados.0.smithi142.stdout:361: writing smithi14230322-48 from 1146880 to 1572864 tid 2 2024-04-18T23:12:04.581 INFO:tasks.rados.rados.0.smithi142.stdout:361: writing smithi14230322-48 from 1572864 to 2211840 tid 3 2024-04-18T23:12:04.586 INFO:tasks.rados.rados.0.smithi142.stdout:361: writing smithi14230322-48 from 2211840 to 2703360 tid 4 2024-04-18T23:12:04.586 INFO:tasks.rados.rados.0.smithi142.stdout:362: copy_from oid 19 from oid 10 current snap is 35 2024-04-18T23:12:04.586 INFO:tasks.rados.rados.0.smithi142.stdout:363: rmattr oid 18 current snap is 35 2024-04-18T23:12:04.586 INFO:tasks.rados.rados.0.smithi142.stdout:363: done (7 left) 2024-04-18T23:12:04.586 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 41 current snap is 35 2024-04-18T23:12:04.586 INFO:tasks.rados.rados.0.smithi142.stdout:364: seq_num 132 ranges {0=671744,671744=573440,1245184=753664,1998848=229376} 2024-04-18T23:12:04.590 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:12:04.590 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:12:04.593 INFO:tasks.rados.rados.0.smithi142.stdout:364: writing smithi14230322-41 from 0 to 671744 tid 1 2024-04-18T23:12:04.599 INFO:tasks.rados.rados.0.smithi142.stdout:364: writing smithi14230322-41 from 671744 to 1245184 tid 2 2024-04-18T23:12:04.606 INFO:tasks.rados.rados.0.smithi142.stdout:364: writing smithi14230322-41 from 1245184 to 1998848 tid 3 2024-04-18T23:12:04.609 INFO:tasks.rados.rados.0.smithi142.stdout:364: writing smithi14230322-41 from 1998848 to 2228224 tid 4 2024-04-18T23:12:04.609 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 24 current snap is 35 2024-04-18T23:12:04.609 INFO:tasks.rados.rados.0.smithi142.stdout:365: seq_num 133 ranges {0=688128,688128=720896,1409024=475136,1884160=360448} 2024-04-18T23:12:04.613 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:12:04.613 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:12:04.615 INFO:tasks.rados.rados.0.smithi142.stdout:365: writing smithi14230322-24 from 0 to 688128 tid 1 2024-04-18T23:12:04.621 INFO:tasks.rados.rados.0.smithi142.stdout:365: writing smithi14230322-24 from 688128 to 1409024 tid 2 2024-04-18T23:12:04.625 INFO:tasks.rados.rados.0.smithi142.stdout:365: writing smithi14230322-24 from 1409024 to 1884160 tid 3 2024-04-18T23:12:04.628 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:04.627+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:04.628 INFO:tasks.rados.rados.0.smithi142.stdout:365: writing smithi14230322-24 from 1884160 to 2244608 tid 4 2024-04-18T23:12:04.628 INFO:tasks.rados.rados.0.smithi142.stdout:366: copy_from oid 11 from oid 12 current snap is 35 2024-04-18T23:12:04.628 INFO:tasks.rados.rados.0.smithi142.stdout:354: finishing rollback tid 0 to smithi14230322-37 2024-04-18T23:12:04.628 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 35 current snap is 35 2024-04-18T23:12:04.629 INFO:tasks.rados.rados.0.smithi142.stdout:367: seq_num 134 ranges {2981888=704512,3686400=475136,4161536=573440,4734976=507904} 2024-04-18T23:12:04.637 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:12:04.637 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:12:04.638 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:04.638 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:04.639 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:12:04.639 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:04.652 INFO:tasks.rados.rados.0.smithi142.stdout:367: writing smithi14230322-35 from 2981888 to 3686400 tid 1 2024-04-18T23:12:04.653 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:12:04.655 INFO:tasks.rados.rados.0.smithi142.stdout:367: writing smithi14230322-35 from 3686400 to 4161536 tid 2 2024-04-18T23:12:04.658 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:12:04.658 INFO:tasks.rados.rados.0.smithi142.stdout:367: writing smithi14230322-35 from 4161536 to 4734976 tid 3 2024-04-18T23:12:04.658 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:12:04.661 INFO:tasks.rados.rados.0.smithi142.stdout:367: writing smithi14230322-35 from 4734976 to 5242880 tid 4 2024-04-18T23:12:04.661 INFO:tasks.rados.rados.0.smithi142.stdout:368: snap_remove snap 33 2024-04-18T23:12:04.676 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:12:04.676 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:12:04.697 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:12:04.698 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:12:04.717 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:12:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:12:04.728 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:04.726+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:04.742 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:12:04.742 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:12:04.777 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:12:04.778 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:12:04.778 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:04.792 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:12:04.830 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:04.826+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:04.920 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:04.920 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:12:04.920 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:12:04.920 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:12:04.920 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:133 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'91, mlcod=28'85)", 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 75.641523813000006, 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20553276400000001, 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.921 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277424+0000", 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.21e-06 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277436+0000", 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1693e-05 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.922 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277464+0000", 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8608000000000001e-05 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377595+0000", 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100130684 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377730+0000", 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013455499999999999 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.923 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377781+0000", 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1406999999999998e-05 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482870+0000", 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105089222 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482955+0000", 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4384999999999999e-05 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.924 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:155 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'109, mlcod=28'105)", 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.783985702999999, 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20593233399999999, 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.925 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134962+0000", 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8560000000000001e-06 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.926 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135074+0000", 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112357 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135099+0000", 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4705000000000001e-05 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135115+0000", 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5855999999999999e-05 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.927 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135494+0000", 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037909800000000002 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135551+0000", 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7176999999999998e-05 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340852+0000", 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20530157900000001 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.928 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340892+0000", 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9706000000000002e-05 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:157 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'111, mlcod=28'105)", 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 74.782464383999994, 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20611499799999999, 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:12:04.929 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136483+0000", 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.576e-06 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136499+0000", 2024-04-18T23:12:04.930 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6529e-05 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136520+0000", 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0571999999999999e-05 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136532+0000", 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.223e-05 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.931 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237736+0000", 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10120357200000001 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237776+0000", 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0397e-05 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342561+0000", 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104784615 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.932 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342596+0000", 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5506999999999997e-05 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:201 3.es0 3:78aee268:::smithi14230322-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.357189965000003, 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41057925299999998, 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.933 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.934 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561757+0000", 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.47e-07 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561762+0000", 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.04e-06 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561777+0000", 2024-04-18T23:12:04.935 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4962e-05 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561785+0000", 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4500000000000004e-06 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562118+0000", 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000332882 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562139+0000", 2024-04-18T23:12:04.936 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1101999999999999e-05 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.769814+0000", 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.207675318 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770164+0000", 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034956199999999998 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.972335+0000", 2024-04-18T23:12:04.937 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20217098999999999 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.4.21:1 3.es0 3:7a4badf6:::smithi14230322-17:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 64.157413132000002, 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31203379199999998, 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.938 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761535+0000", 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7999999999999999e-06 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.939 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761547+0000", 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1279e-05 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761568+0000", 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1549e-05 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761588+0000", 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0426000000000001e-05 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.940 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770853+0000", 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092644660000000007 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.771105+0000", 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025160800000000001 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073566+0000", 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30246166400000002 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.941 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:333 3.es0 3:799dcc49:::smithi14230322-39:head [create,setxattr Yjk]k``VRNVkfm]Rnjieeerecjk`Wm (15648) in=30b,setxattr QgpqijWTrohYWOsYikii[YdsSee`QrtrZ]nSPTUt (1225) in=40b,setxattr l]iRdbVhahjiOeVf`Yc\\U_OYjY]nr[js\\NggsVVY (1015) in=40b,setxattr b^W[ZgXWiRmfON^`rihabiO]lNkro`P (15648) in=31b,setxattr VReWeRiYgNSZU\\d\\YOrNS`]ogtdOjVijWVeY`fhd (1051) in=40b,setxattr SYsfepqPdsc`cUg\\WaTfgrcUYhUbTiaQY_`rlt (15648) in=38b,setxattr P[`jheVP[fjpnXqdjPhddi_oWUf\\dWWbSlodc`eS (1086) in=40b,setxattr X[^VkT`QVNea_jfmlVkdafY]hrorgVhl]fqtiVie (1226) in=40b,setxattr Xs\\WjfX^_Ss_RPr_tT^a]fnSSlYrcPTbmSaksi_c (19437) in=40b,setxattr QUn_^p]__nRhn\\srj]jla^iQSPOpgQcVcQZU^ool (1823) in=40b,setxattr f\\^WXdjheR\\aQSYk]Un_^hkrZTSVdhiZqgcin[Sa (1155) in=40b,setxattr bkb]mNeZddhQrcsORPSgqbWecOTtcPYZaRQOg`nY (1614) in=40b,setxattr _T_atc\\\\Rml^aOPkNsbsb^VatOWSs\\URSjVqfe\\Z (1226) in=40b,setxattr obYqlk]hgNOQgXa^ZhalmPcXl\\lkOdNNiYlOfdhq (1612) in=40b,setxattr VsbnoacTrXVpb]PV]kVPYbQP_qefUdfOgk`qUaNa (1999) in=40b,setxattr ZqTlaiiPajkNT^ce\\W\\oklalkiT`\\\\nhjcgpZrUa (3297) in=40b,setxattr QfUmVl\\][PRg\\ajWQS`]Oe^Zlig]b]Rs_o]NsY[i (244) in=40b,setxattr R_Ucr`a\\j]Ri_YSp[gamgRSRd`]UYnPRspctZ`R] (4068) in=40b,setxattr aj`m[hpi__Rrg_Z[m^rVOVrVgk^cS_]piWV]lkRZ (947) in=40b,setxattr ^nWmd_TeciTqUOnWXn_jXch]heNac[WdcqdPbiT^ (1647) in=40b] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.291589770000002, 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30962595399999998, 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:04.942 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627359+0000", 2024-04-18T23:12:04.943 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3400000000000002e-06 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627676+0000", 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000317144 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627710+0000", 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3855e-05 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627753+0000", 2024-04-18T23:12:04.944 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3341999999999999e-05 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627926+0000", 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017248799999999999 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627946+0000", 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9868999999999999e-05 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729527+0000", 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101581587 2024-04-18T23:12:04.945 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729747+0000", 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021955299999999999 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.733259+0000", 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035115820000000001 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739147+0000", 2024-04-18T23:12:04.946 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058886169999999996 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936726+0000", 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.197578422 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936982+0000", 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025615499999999999 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.947 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:344 3.es0 3:7a4badf6:::smithi14230322-17:head [delete] snapc 8=[7,4] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 52.277531213000003, 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.295980827, 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 344 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:12:04.948 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641417+0000", 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2630000000000002e-06 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.949 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641427+0000", 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0674e-05 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641460+0000", 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2768999999999997e-05 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641482+0000", 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1959000000000001e-05 2024-04-18T23:12:04.950 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641642+0000", 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016009100000000001 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641653+0000", 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.128e-05 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735170+0000", 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.093517063999999997 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.951 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735466+0000", 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029521400000000002 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.736750+0000", 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012846540000000001 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739533+0000", 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027831589999999999 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.952 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937242+0000", 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19770827199999999 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937395+0000", 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000153428 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 56/49 ECSubWrite(tid=94, reqid=client.4568.0:421, at_version=56'16, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.275560794, 2024-04-18T23:12:04.953 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.27351094799999998, 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.954 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643386+0000", 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4300000000000002e-07 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643802+0000", 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000416136 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643934+0000", 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013215300000000001 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.955 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643959+0000", 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4973e-05 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.812936+0000", 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.168977035 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.812952+0000", 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6419e-05 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916818+0000", 2024-04-18T23:12:04.956 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10386601099999999 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916896+0000", 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7378000000000004e-05 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 56/49 ECSubWrite(tid=95, reqid=client.4568.0:422, at_version=56'17, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.269945544000002, 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26796196300000003, 2024-04-18T23:12:04.957 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.958 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649002+0000", 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8199999999999999e-06 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649359+0000", 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035705699999999998 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649390+0000", 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.146e-05 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.959 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749486+0000", 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100095925 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.814696+0000", 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.065209634000000002 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.814713+0000", 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7082999999999999e-05 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.960 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916904+0000", 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10219086500000001 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916962+0000", 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8118999999999997e-05 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:467 3.es0 3:799dcc49:::smithi14230322-39:head [read 0~638976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.365304084000002, 2024-04-18T23:12:04.961 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39041841700000002, 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 467 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.962 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553642+0000", 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3000000000000001e-07 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.963 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553645+0000", 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.813e-06 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553650+0000", 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6840000000000001e-06 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553655+0000", 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.964 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654393+0000", 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100738487 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654401+0000", 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9340000000000006e-06 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.657013+0000", 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026115800000000001 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.965 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.657158+0000", 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014565499999999999 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.944060+0000", 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28690162600000002 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=42, last_complete=61'29, committed=1, applied=1))", 2024-04-18T23:12:04.966 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 38.159491170000003, 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28503450299999999, 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.967 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759455+0000", 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7100000000000001e-07 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759459+0000", 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1760000000000003e-06 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759471+0000", 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1523e-05 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.968 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.859553+0000", 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10008174 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.044450+0000", 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18489760999999999 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.044489+0000", 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8683000000000002e-05 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.969 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=44, last_complete=61'31, committed=1, applied=1))", 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.559742788999998, 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29027006799999999, 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.970 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359204+0000", 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.282e-06 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359210+0000", 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6579999999999999e-06 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359227+0000", 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7283999999999999e-05 2024-04-18T23:12:04.971 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359257+0000", 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9643000000000001e-05 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649393+0000", 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29013631200000001 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649473+0000", 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9888999999999994e-05 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.972 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=45, last_complete=61'33, committed=1, applied=1))", 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.558611548999998, 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28925603500000002, 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.973 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360335+0000", 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8999999999999995e-07 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360340+0000", 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4160000000000003e-06 2024-04-18T23:12:04.974 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360354+0000", 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3709999999999999e-05 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360362+0000", 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6120000000000001e-06 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649559+0000", 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28919758400000001 2024-04-18T23:12:04.975 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649590+0000", 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0822999999999999e-05 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:497 3.es0 3:799dcc49:::smithi14230322-39:head [rmxattr QgpqijWTrohYWOsYikii[YdsSee`QrtrZ]nSPTUt in=40b,rmxattr R_Ucr`a\\j]Ri_YSp[gamgRSRd`]UYnPRspctZ`R] in=40b,rmxattr VReWeRiYgNSZU\\d\\YOrNS`]ogtdOjVijWVeY`fhd in=40b,rmxattr X[^VkT`QVNea_jfmlVkdafY]hrorgVhl]fqtiVie in=40b,rmxattr _T_atc\\\\Rml^aOPkNsbsb^VatOWSs\\URSjVqfe\\Z in=40b,rmxattr bkb]mNeZddhQrcsORPSgqbWecOTtcPYZaRQOg`nY in=40b] snapc 15=[15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.388308610000003, 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21045935599999999, 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.976 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 497 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.977 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530641+0000", 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7869999999999999e-06 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530672+0000", 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1642e-05 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.978 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530707+0000", 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4566000000000001e-05 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530726+0000", 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9130000000000001e-05 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530864+0000", 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138307 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:12:04.979 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530878+0000", 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.379e-05 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.534931+0000", 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004053035 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.535122+0000", 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190612 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.536668+0000", 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015465800000000001 2024-04-18T23:12:04.980 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.640407+0000", 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103738334 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.740945+0000", 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100538086 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.741096+0000", 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015148700000000001 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.981 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:551 3.es0 3:78aee268:::smithi14230322-13:head [append 0~425984] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.563216+0000", 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.355729397000001, 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31486470599999999, 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 551 2024-04-18T23:12:04.982 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.563216+0000", 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.563216+0000", 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.983 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.563219+0000", 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.92e-06 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564232+0000", 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010124229999999999 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564266+0000", 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4282000000000001e-05 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.984 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564286+0000", 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0134000000000001e-05 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564452+0000", 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000165876 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564647+0000", 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019565699999999999 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.566327+0000", 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001679313 2024-04-18T23:12:04.985 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.568134+0000", 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018070339999999999 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.877907+0000", 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30977282 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878081+0000", 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017424699999999999 2024-04-18T23:12:04.986 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:552 3.es0 3:78aee268:::smithi14230322-13:head [append 0~786432] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.570285+0000", 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.348660321000001, 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30865453999999998, 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:04.987 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 552 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.570285+0000", 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.570285+0000", 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.570295+0000", 2024-04-18T23:12:04.988 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3340000000000008e-06 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.571762+0000", 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001467007 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.571793+0000", 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1303000000000002e-05 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.571808+0000", 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4853e-05 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.989 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.670635+0000", 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098827154 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.670823+0000", 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018851999999999999 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.672550+0000", 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017261030000000001 2024-04-18T23:12:04.990 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.777055+0000", 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10450546500000001 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878818+0000", 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10176346999999999 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878940+0000", 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012133099999999999 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.991 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:553 3.es0 3:78aee268:::smithi14230322-13:head [append 0~376832] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.573318+0000", 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.345627091000001, 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40648862800000002, 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 553 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.992 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.573318+0000", 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.573318+0000", 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.573320+0000", 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5239999999999999e-06 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.993 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574342+0000", 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010218429999999999 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574366+0000", 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3689999999999998e-05 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574378+0000", 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2386e-05 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.994 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.774530+0000", 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20015175199999999 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.774703+0000", 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017317299999999999 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.776150+0000", 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014476020000000001 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.995 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878103+0000", 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101952266 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.979668+0000", 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101564918 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.979807+0000", 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139474 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.996 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:554 3.es0 3:78aee268:::smithi14230322-13:head [setxattr _header (57) in=7b] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.574400+0000", 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.344545655000001, 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50849109599999998, 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 554 2024-04-18T23:12:04.997 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574400+0000", 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574400+0000", 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574401+0000", 2024-04-18T23:12:04.998 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.203e-06 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574407+0000", 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.207e-06 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574418+0000", 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0257e-05 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574425+0000", 2024-04-18T23:12:04.999 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7319999999999998e-06 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.879281+0000", 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30485609800000002 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.879453+0000", 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017140699999999999 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.880371+0000", 2024-04-18T23:12:05.000 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091843199999999995 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.881707+0000", 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001335943 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.082730+0000", 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201023233 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.082891+0000", 2024-04-18T23:12:05.001 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016058399999999999 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:555 3.es0 3:78aee268:::smithi14230322-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.574436+0000", 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.344509085999999, 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81095218999999996, 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:05.002 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 555 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574436+0000", 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574436+0000", 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.003 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574437+0000", 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7700000000000004e-07 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574441+0000", 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574451+0000", 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.012e-05 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.674559+0000", 2024-04-18T23:12:05.004 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100107769 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.880898+0000", 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20633925 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.881048+0000", 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014995699999999999 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.083603+0000", 2024-04-18T23:12:05.005 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202554858 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.083750+0000", 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014693 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.385389+0000", 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30163906899999998 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.006 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:683 3.es0 3:78aee268:::smithi14230322-13:head [read 0~4669440,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:49.796228+0000", 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.122717331, 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31501509300000002, 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 683 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.007 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796228+0000", 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796228+0000", 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796229+0000", 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7899999999999998e-07 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.008 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796234+0000", 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5180000000000004e-06 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796246+0000", 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1525000000000001e-05 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796253+0000", 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9870000000000002e-06 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.009 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796466+0000", 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021326400000000001 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796477+0000", 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.06e-05 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.801736+0000", 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052596689999999998 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.010 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.801835+0000", 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8825000000000003e-05 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.111243+0000", 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30940812600000001 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.011 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:05.013 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:12:05.013 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:05.010+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:05.114 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:05.112+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:05.141 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:05.141 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:12:05.141 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:05.155 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:12:05.215 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:05.210+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:05.315 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:05.314+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:05.324 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:12:05.324 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:12:05.324 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:05.340 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:12:05.416 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:05.416+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:05.463 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:12:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:12:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:12:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=158, reqid=client.4568.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:12:05.463 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.835404564000001, 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203058724, 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.464 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627730+0000", 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.606e-06 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627841+0000", 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110692 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627889+0000", 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8211000000000003e-05 2024-04-18T23:12:05.465 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627919+0000", 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9961999999999999e-05 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629700+0000", 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001780963 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629730+0000", 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0082000000000001e-05 2024-04-18T23:12:05.466 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830736+0000", 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201005607 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830787+0000", 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1601000000000003e-05 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.467 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=159, reqid=client.4568.0:25, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.835130750999994, 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20284285199999999, 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.468 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628004+0000", 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0270000000000001e-06 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628137+0000", 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013336000000000001 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.469 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628197+0000", 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0504000000000002e-05 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628221+0000", 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3037999999999999e-05 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728171+0000", 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099950707999999999 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.470 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728190+0000", 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8709e-05 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830795+0000", 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102605138 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830845+0000", 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0368000000000003e-05 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.471 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=160, reqid=client.4568.0:26, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.834876030000004, 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20264686400000001, 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.472 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628258+0000", 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628292+0000", 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4054999999999998e-05 2024-04-18T23:12:05.473 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628317+0000", 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5035999999999999e-05 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628338+0000", 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1058999999999999e-05 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728828+0000", 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100489521 2024-04-18T23:12:05.474 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728846+0000", 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8419000000000001e-05 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830850+0000", 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10200397 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830904+0000", 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3844000000000002e-05 2024-04-18T23:12:05.475 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=161, reqid=client.4568.0:31, at_version=34'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.834741985999997, 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202594633, 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.476 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628392+0000", 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6000000000000004e-07 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.477 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628541+0000", 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149386 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628572+0000", 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0975000000000001e-05 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628594+0000", 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1608999999999999e-05 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.478 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829299+0000", 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20070527499999999 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829314+0000", 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4718999999999999e-05 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830915+0000", 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001601412 2024-04-18T23:12:05.479 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830986+0000", 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0697000000000004e-05 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 46/31 ECSubWrite(tid=20, reqid=client.4568.0:331, at_version=46'14, trim_to=0'0, roll_forward_to=40'12))", 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:11.432076+0000", 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.031057296, 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20163852800000001, 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.480 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432076+0000", 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432076+0000", 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432077+0000", 2024-04-18T23:12:05.481 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1399999999999997e-07 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432118+0000", 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1155999999999997e-05 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432134+0000", 2024-04-18T23:12:05.482 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6198e-05 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.432147+0000", 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3385e-05 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.632376+0000", 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200228505 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.632385+0000", 2024-04-18T23:12:05.483 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2879999999999998e-06 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.633631+0000", 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001246268 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:11.633715+0000", 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3114000000000004e-05 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.484 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 56/31 ECSubWrite(tid=164, reqid=client.4568.0:431, at_version=56'73, trim_to=0'0, roll_forward_to=55'71))", 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.917621445000002, 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20358216400000001, 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.485 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7300000000000002e-07 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545582+0000", 2024-04-18T23:12:05.486 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9449000000000005e-05 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545607+0000", 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5741999999999998e-05 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545625+0000", 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7503000000000002e-05 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.645851+0000", 2024-04-18T23:12:05.487 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100225733 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.645870+0000", 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8972999999999999e-05 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749022+0000", 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103152813 2024-04-18T23:12:05.488 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749094+0000", 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1577999999999998e-05 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s6 70/68 ECSubWrite(tid=80, reqid=client.4568.0:549, at_version=0'0, trim_to=0'0, roll_forward_to=69'41))", 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:37.152355+0000", 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.310778454000001, 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200897031, 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.489 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152355+0000", 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152355+0000", 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.490 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152357+0000", 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.784e-06 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152384+0000", 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7650000000000001e-05 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152417+0000", 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2422000000000002e-05 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.491 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152440+0000", 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3260000000000001e-05 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152689+0000", 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024853199999999998 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152709+0000", 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0006000000000001e-05 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.492 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.353193+0000", 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200484738 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.353252+0000", 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8638999999999999e-05 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s6 70/68 ECSubWrite(tid=81, reqid=client.4568.0:550, at_version=70'42, trim_to=0'0, roll_forward_to=69'41))", 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:37.152485+0000", 2024-04-18T23:12:05.493 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 28.310648689000001, 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201685279, 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152485+0000", 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152485+0000", 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.494 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152486+0000", 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0999999999999997e-07 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152536+0000", 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0312000000000002e-05 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152560+0000", 2024-04-18T23:12:05.495 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4046e-05 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152580+0000", 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9792999999999998e-05 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.352861+0000", 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20028146199999999 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.352877+0000", 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5420000000000001e-05 2024-04-18T23:12:05.496 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.354095+0000", 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012185469999999999 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.354170+0000", 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4789000000000005e-05 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.497 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 71/70 ECSubWrite(tid=35, reqid=osd.5.0:35, at_version=71'26, trim_to=0'0, roll_forward_to=71'22))", 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.092364+0000", 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.370769276000001, 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20238825899999999, 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092364+0000", 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.498 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092364+0000", 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092365+0000", 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.373e-06 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092404+0000", 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8241999999999997e-05 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.499 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092428+0000", 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4729999999999999e-05 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092443+0000", 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4883e-05 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.292777+0000", 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200333228 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.500 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.292791+0000", 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4059e-05 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.294674+0000", 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001883205 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.294752+0000", 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8539000000000001e-05 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.501 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 72/70 ECSubWrite(tid=40, reqid=osd.5.0:39, at_version=0'0, trim_to=0'0, roll_forward_to=72'30))", 2024-04-18T23:12:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:39.022976+0000", 2024-04-18T23:12:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.440157404000001, 2024-04-18T23:12:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20188773400000001, 2024-04-18T23:12:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.502 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.022976+0000", 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.022976+0000", 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.022978+0000", 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6169999999999999e-06 2024-04-18T23:12:05.503 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.023000+0000", 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2660999999999999e-05 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.023038+0000", 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8104e-05 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.023054+0000", 2024-04-18T23:12:05.504 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6062e-05 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.123257+0000", 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10020230199999999 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.123276+0000", 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8842999999999999e-05 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.224763+0000", 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101487857 2024-04-18T23:12:05.505 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.224864+0000", 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100288 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s5 72/71 ECSubWrite(tid=149, reqid=osd.14.0:149, at_version=72'26, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:39.024690+0000", 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 26.438443256999999, 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20278465600000001, 2024-04-18T23:12:05.506 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024690+0000", 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024690+0000", 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.507 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024692+0000", 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024756+0000", 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4202000000000006e-05 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024796+0000", 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9762999999999999e-05 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.508 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024821+0000", 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4688000000000001e-05 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.223945+0000", 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19912484499999999 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.223959+0000", 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3985000000000001e-05 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.509 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.227409+0000", 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003449428 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.227475+0000", 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5932000000000005e-05 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s3 77/67 ECSubRead(tid=217, to_read={3:42c34e84:::smithi14230322-11:head=0,458752,0}, subchunks={3:42c34e84:::smithi14230322-11:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:43.843070+0000", 2024-04-18T23:12:05.510 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.620063123000001, 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30067776099999999, 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843070+0000", 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843070+0000", 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.511 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843072+0000", 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6169999999999999e-06 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843087+0000", 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843118+0000", 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0859999999999999e-05 2024-04-18T23:12:05.512 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843143+0000", 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4766e-05 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.143457+0000", 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30031421600000002 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.143748+0000", 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029077999999999998 2024-04-18T23:12:05.513 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 78/31 ECSubWrite(tid=264, reqid=osd.4.0:260, at_version=0'0, trim_to=0'0, roll_forward_to=78'134))", 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.818594+0000", 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.644539229999999, 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20360535399999999, 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.514 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818594+0000", 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818594+0000", 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818596+0000", 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8870000000000001e-06 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.515 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818629+0000", 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3408000000000003e-05 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818665+0000", 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5160000000000002e-05 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818689+0000", 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4655e-05 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818836+0000", 2024-04-18T23:12:05.516 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014703000000000001 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818853+0000", 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6636000000000001e-05 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022114+0000", 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203261577 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022199+0000", 2024-04-18T23:12:05.517 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5000999999999995e-05 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 78/31 ECSubWrite(tid=265, reqid=osd.4.0:265, at_version=78'136, trim_to=0'0, roll_forward_to=78'134))", 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.819697+0000", 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 20.643436092000002, 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202551706, 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.518 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819697+0000", 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819697+0000", 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819698+0000", 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.24e-06 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.519 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819756+0000", 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7926000000000001e-05 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819792+0000", 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5364000000000002e-05 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819815+0000", 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2861999999999999e-05 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.520 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.920043+0000", 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100228418 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.920060+0000", 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6769000000000002e-05 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022212+0000", 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10215207 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.521 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022249+0000", 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7057000000000001e-05 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 83/31 ECSubWrite(tid=326, reqid=client.4568.0:730, at_version=83'170, trim_to=0'0, roll_forward_to=82'166))", 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.355423+0000", 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.107710540999999, 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202790049, 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.522 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355423+0000", 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355423+0000", 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355423+0000", 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4000000000000002e-07 2024-04-18T23:12:05.523 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355502+0000", 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9090000000000003e-05 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355511+0000", 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1779999999999992e-06 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355526+0000", 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4688999999999999e-05 2024-04-18T23:12:05.524 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.455687+0000", 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10016045 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.455697+0000", 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0730999999999999e-05 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558181+0000", 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102483502 2024-04-18T23:12:05.525 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558213+0000", 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1968999999999998e-05 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 83/31 ECSubWrite(tid=327, reqid=client.4568.0:731, at_version=83'171, trim_to=0'0, roll_forward_to=82'166))", 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.355547+0000", 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.107586747999999, 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20268904900000001, 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.526 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355547+0000", 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355547+0000", 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.527 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355547+0000", 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999998e-07 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355623+0000", 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6070999999999997e-05 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355634+0000", 2024-04-18T23:12:05.528 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1194e-05 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355643+0000", 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6920000000000008e-06 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.456260+0000", 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100616868 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.456290+0000", 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9867999999999999e-05 2024-04-18T23:12:05.529 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558218+0000", 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10192849900000001 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558236+0000", 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7493e-05 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.530 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 83/31 ECSubWrite(tid=328, reqid=client.4568.0:732, at_version=83'172, trim_to=0'0, roll_forward_to=83'168))", 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.355659+0000", 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.107474570000001, 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20262286099999999, 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355659+0000", 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.531 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355659+0000", 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355659+0000", 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7399999999999999e-07 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355705+0000", 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6103000000000002e-05 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.532 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355717+0000", 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1939e-05 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355726+0000", 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1710000000000008e-06 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.557073+0000", 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20134684799999999 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.533 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.557091+0000", 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7858000000000001e-05 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558262+0000", 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011712820000000001 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558282+0000", 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9286000000000001e-05 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.534 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 83/31 ECSubWrite(tid=329, reqid=client.4568.0:733, at_version=83'173, trim_to=0'0, roll_forward_to=83'168))", 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.355740+0000", 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.107393817, 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202871142, 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355740+0000", 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.535 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355740+0000", 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355740+0000", 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.53e-07 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355752+0000", 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2571999999999999e-05 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.536 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355762+0000", 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5529999999999998e-06 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355771+0000", 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0329999999999997e-06 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.557972+0000", 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20220071000000001 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.537 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.557981+0000", 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4590000000000004e-06 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558569+0000", 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00058762599999999995 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558611+0000", 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1835999999999997e-05 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.538 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs4 94/31 ECSubWrite(tid=405, reqid=osd.4.0:405, at_version=94'167, trim_to=0'0, roll_forward_to=92'163))", 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:00.559314+0000", 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9038196259999998, 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20159027400000001, 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.559314+0000", 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.539 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.559314+0000", 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.559315+0000", 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.584e-06 2024-04-18T23:12:05.540 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.559356+0000", 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1208000000000003e-05 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.559385+0000", 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8354999999999999e-05 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.559401+0000", 2024-04-18T23:12:05.541 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5835999999999999e-05 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560563+0000", 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011627149999999999 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560584+0000", 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0239000000000001e-05 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.542 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.760843+0000", 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20025896600000001 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.760904+0000", 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1371000000000006e-05 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs4 94/31 ECSubWrite(tid=406, reqid=client.4568.0:826, at_version=94'168, trim_to=0'0, roll_forward_to=92'163))", 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:00.560036+0000", 2024-04-18T23:12:05.543 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 4.9030970519999997, 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20184205999999999, 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560036+0000", 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560036+0000", 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:05.544 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560038+0000", 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.624e-06 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560395+0000", 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035757699999999998 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560419+0000", 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3462e-05 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.545 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560433+0000", 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.393e-05 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.759783+0000", 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19934999 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.759799+0000", 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6370000000000001e-05 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.546 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.761815+0000", 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020154130000000002 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.761878+0000", 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3694000000000004e-05 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:05.547 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:05.549 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:12:05.550 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:12:05.546+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:12:05.573 INFO:tasks.rados.rados.0.smithi142.stdout:368: done (11 left) 2024-04-18T23:12:05.573 INFO:tasks.rados.rados.0.smithi142.stdout:369: copy_from oid 49 from oid 16 current snap is 35 2024-04-18T23:12:05.573 INFO:tasks.rados.rados.0.smithi142.stdout:366: finishing copy_from racing read to smithi14230322-11 2024-04-18T23:12:05.573 INFO:tasks.rados.rados.0.smithi142.stdout:365: finishing write tid 1 to smithi14230322-24 2024-04-18T23:12:05.573 INFO:tasks.rados.rados.0.smithi142.stdout:354: finishing rollback tid 1 to smithi14230322-37 2024-04-18T23:12:05.573 INFO:tasks.rados.rados.0.smithi142.stdout:354: finishing rollback tid 2 to smithi14230322-37 2024-04-18T23:12:05.573 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 37 v 224 (ObjNum 85 snap 20 seq_num 85) dirty exists 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:359: finishing rollback tid 0 to smithi14230322-42 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:359: finishing rollback tid 1 to smithi14230322-42 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:359: finishing rollback tid 2 to smithi14230322-42 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 42 v 139 (ObjNum 109 snap 25 seq_num 109) dirty exists 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 20 v 183 (ObjNum 95 snap 21 seq_num 95) dirty exists 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:366: finishing copy_from to smithi14230322-11 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 11 v 4 (ObjNum 7 snap 0 seq_num 7) dirty exists 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:365: finishing write tid 2 to smithi14230322-24 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:365: finishing write tid 3 to smithi14230322-24 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:365: finishing write tid 4 to smithi14230322-24 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:365: finishing write tid 5 to smithi14230322-24 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:355: finishing write tid 1 to smithi14230322-14 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:361: finishing write tid 1 to smithi14230322-48 2024-04-18T23:12:05.574 INFO:tasks.rados.rados.0.smithi142.stdout:365: finishing write tid 6 to smithi14230322-24 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:365: oid 24 updating version 0 to 114 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:365: oid 24 version 114 is already newer than 113 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:365: oid 24 updating version 114 to 116 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:365: oid 24 updating version 116 to 117 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:365: oid 24 version 117 is already newer than 115 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 24 v 117 (ObjNum 133 snap 35 seq_num 133) dirty exists 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:365: left oid 24 (ObjNum 133 snap 35 seq_num 133) 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:361: finishing write tid 2 to smithi14230322-48 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:354: done (11 left) 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:359: done (10 left) 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:360: done (9 left) 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:365: done (8 left) 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:366: done (7 left) 2024-04-18T23:12:05.575 INFO:tasks.rados.rados.0.smithi142.stdout:370: copy_from oid 5 from oid 32 current snap is 35 2024-04-18T23:12:05.576 INFO:tasks.rados.rados.0.smithi142.stdout:371: snap_create 2024-04-18T23:12:05.581 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T23:12:05.649 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:05.648+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:05.656 INFO:tasks.rados.rados.0.smithi142.stdout:362: finishing copy_from racing read to smithi14230322-19 2024-04-18T23:12:05.664 INFO:tasks.rados.rados.0.smithi142.stdout:362: finishing copy_from to smithi14230322-19 2024-04-18T23:12:05.664 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 19 v 104 (ObjNum 107 snap 25 seq_num 107) dirty exists 2024-04-18T23:12:05.679 INFO:tasks.rados.rados.0.smithi142.stdout:361: finishing write tid 3 to smithi14230322-48 2024-04-18T23:12:05.679 INFO:tasks.rados.rados.0.smithi142.stdout:361: finishing write tid 4 to smithi14230322-48 2024-04-18T23:12:05.724 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:05.724 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:12:05.725 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:05.737 INFO:tasks.rados.rados.0.smithi142.stdout:355: finishing write tid 2 to smithi14230322-14 2024-04-18T23:12:05.737 INFO:tasks.rados.rados.0.smithi142.stdout:355: finishing write tid 3 to smithi14230322-14 2024-04-18T23:12:05.737 INFO:tasks.rados.rados.0.smithi142.stdout:355: finishing write tid 4 to smithi14230322-14 2024-04-18T23:12:05.737 INFO:tasks.rados.rados.0.smithi142.stdout:355: finishing write tid 5 to smithi14230322-14 2024-04-18T23:12:05.737 INFO:tasks.rados.rados.0.smithi142.stdout:357: finishing rollback tid 0 to smithi14230322-36 2024-04-18T23:12:05.737 INFO:tasks.rados.rados.0.smithi142.stdout:367: finishing write tid 1 to smithi14230322-35 2024-04-18T23:12:05.738 INFO:tasks.rados.rados.0.smithi142.stdout:367: finishing write tid 2 to smithi14230322-35 2024-04-18T23:12:05.738 INFO:tasks.rados.rados.0.smithi142.stdout:367: finishing write tid 3 to smithi14230322-35 2024-04-18T23:12:05.739 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:12:05.750 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:05.746+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:05.772 INFO:tasks.rados.rados.0.smithi142.stdout:361: finishing write tid 5 to smithi14230322-48 2024-04-18T23:12:05.773 INFO:tasks.rados.rados.0.smithi142.stdout:364: finishing write tid 1 to smithi14230322-41 2024-04-18T23:12:05.774 INFO:tasks.rados.rados.0.smithi142.stdout:364: finishing write tid 2 to smithi14230322-41 2024-04-18T23:12:05.774 INFO:tasks.rados.rados.0.smithi142.stdout:364: finishing write tid 3 to smithi14230322-41 2024-04-18T23:12:05.775 INFO:tasks.rados.rados.0.smithi142.stdout:364: finishing write tid 4 to smithi14230322-41 2024-04-18T23:12:05.775 INFO:tasks.rados.rados.0.smithi142.stdout:364: finishing write tid 5 to smithi14230322-41 2024-04-18T23:12:05.838 INFO:tasks.rados.rados.0.smithi142.stdout:367: finishing write tid 4 to smithi14230322-35 2024-04-18T23:12:05.839 INFO:tasks.rados.rados.0.smithi142.stdout:357: finishing rollback tid 1 to smithi14230322-36 2024-04-18T23:12:05.839 INFO:tasks.rados.rados.0.smithi142.stdout:367: finishing write tid 5 to smithi14230322-35 2024-04-18T23:12:05.851 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:05.848+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:05.879 INFO:tasks.rados.rados.0.smithi142.stdout:361: finishing write tid 6 to smithi14230322-48 2024-04-18T23:12:05.879 INFO:tasks.rados.rados.0.smithi142.stdout:361: oid 48 updating version 0 to 152 2024-04-18T23:12:05.879 INFO:tasks.rados.rados.0.smithi142.stdout:361: oid 48 updating version 152 to 154 2024-04-18T23:12:05.879 INFO:tasks.rados.rados.0.smithi142.stdout:361: oid 48 version 154 is already newer than 150 2024-04-18T23:12:05.879 INFO:tasks.rados.rados.0.smithi142.stdout:361: oid 48 version 154 is already newer than 153 2024-04-18T23:12:05.879 INFO:tasks.rados.rados.0.smithi142.stdout:361: oid 48 version 154 is already newer than 151 2024-04-18T23:12:05.879 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 48 v 154 (ObjNum 131 snap 35 seq_num 131) dirty exists 2024-04-18T23:12:05.880 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:05.879+0000 7f9f5cbae640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 23 after trimming was started 2024-04-18T23:12:05.880 INFO:tasks.rados.rados.0.smithi142.stdout:361: left oid 48 (ObjNum 131 snap 35 seq_num 131) 2024-04-18T23:12:05.907 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:12:05.907 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:12:05.907 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:05.921 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:12:05.939 INFO:tasks.rados.rados.0.smithi142.stdout:357: finishing rollback tid 2 to smithi14230322-36 2024-04-18T23:12:05.939 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 36 v 234 (ObjNum 56 snap 4 seq_num 56) dirty exists 2024-04-18T23:12:05.952 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:05.950+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:06.046 INFO:tasks.rados.rados.0.smithi142.stdout:369: finishing copy_from racing read to smithi14230322-49 2024-04-18T23:12:06.046 INFO:tasks.rados.rados.0.smithi142.stdout:355: finishing write tid 6 to smithi14230322-14 2024-04-18T23:12:06.046 INFO:tasks.rados.rados.0.smithi142.stdout:355: oid 14 updating version 0 to 227 2024-04-18T23:12:06.046 INFO:tasks.rados.rados.0.smithi142.stdout:355: oid 14 version 227 is already newer than 226 2024-04-18T23:12:06.046 INFO:tasks.rados.rados.0.smithi142.stdout:355: oid 14 updating version 227 to 230 2024-04-18T23:12:06.046 INFO:tasks.rados.rados.0.smithi142.stdout:355: oid 14 version 230 is already newer than 228 2024-04-18T23:12:06.046 INFO:tasks.rados.rados.0.smithi142.stdout:355: oid 14 version 230 is already newer than 229 2024-04-18T23:12:06.046 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 14 v 230 (ObjNum 130 snap 34 seq_num 130) dirty exists 2024-04-18T23:12:06.046 INFO:tasks.rados.rados.0.smithi142.stdout:355: left oid 14 (ObjNum 130 snap 34 seq_num 130) 2024-04-18T23:12:06.053 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:12:06.051+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:12:06.084 INFO:tasks.rados.rados.0.smithi142.stdout:364: finishing write tid 6 to smithi14230322-41 2024-04-18T23:12:06.084 INFO:tasks.rados.rados.0.smithi142.stdout:364: oid 41 updating version 0 to 159 2024-04-18T23:12:06.084 INFO:tasks.rados.rados.0.smithi142.stdout:364: oid 41 version 159 is already newer than 156 2024-04-18T23:12:06.084 INFO:tasks.rados.rados.0.smithi142.stdout:364: oid 41 version 159 is already newer than 157 2024-04-18T23:12:06.084 INFO:tasks.rados.rados.0.smithi142.stdout:364: oid 41 version 159 is already newer than 158 2024-04-18T23:12:06.084 INFO:tasks.rados.rados.0.smithi142.stdout:364: oid 41 version 159 is already newer than 155 2024-04-18T23:12:06.084 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 41 v 159 (ObjNum 132 snap 35 seq_num 132) dirty exists 2024-04-18T23:12:06.085 INFO:tasks.rados.rados.0.smithi142.stdout:364: left oid 41 (ObjNum 132 snap 35 seq_num 132) 2024-04-18T23:12:06.096 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 36/31 ECSubWrite(tid=25, reqid=client.4568.0:185, at_version=36'13, trim_to=0'0, roll_forward_to=35'11))", 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 65.444140150999999, 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103575165, 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.097 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650083+0000", 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.098 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650085+0000", 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8929999999999999e-06 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650164+0000", 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8852999999999999e-05 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.650193+0000", 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9352e-05 2024-04-18T23:12:06.099 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750332+0000", 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100138779 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750511+0000", 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000178679 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.100 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.750538+0000", 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7328999999999999e-05 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753559+0000", 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0030212080000000001 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.753658+0000", 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9072000000000003e-05 2024-04-18T23:12:06.101 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 61/31 ECSubWrite(tid=70, reqid=osd.7.0:70, at_version=61'48, trim_to=0'0, roll_forward_to=58'46))", 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.342282378, 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30382444400000003, 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.102 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.103 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.68e-07 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751960+0000", 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8916e-05 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751973+0000", 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3030999999999999e-05 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.104 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751984+0000", 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0858e-05 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.853770+0000", 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101786394 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.853799+0000", 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8918999999999999e-05 2024-04-18T23:12:06.105 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055725+0000", 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20192524000000001 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055766+0000", 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0917999999999997e-05 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.106 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 61/31 ECSubWrite(tid=71, reqid=osd.7.0:71, at_version=61'50, trim_to=0'0, roll_forward_to=58'46))", 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 39.340738881999997, 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30231752299999998, 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.107 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2699999999999999e-07 2024-04-18T23:12:06.108 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753510+0000", 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4779000000000001e-05 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753524+0000", 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4423999999999999e-05 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.109 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753534+0000", 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7089999999999997e-06 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.954802+0000", 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20126760199999999 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.954819+0000", 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7501999999999999e-05 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.110 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055781+0000", 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10096224299999999 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055802+0000", 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0636999999999999e-05 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.111 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 70/68 ECSubWrite(tid=140, reqid=client.4568.0:546, at_version=70'56, trim_to=0'0, roll_forward_to=69'54))", 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.849944+0000", 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 29.244278991000002, 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10380012, 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.112 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.849944+0000", 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.849944+0000", 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.849947+0000", 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4739999999999999e-06 2024-04-18T23:12:06.113 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.850099+0000", 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0001516 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.850140+0000", 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1832000000000002e-05 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.114 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.850167+0000", 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.6480000000000001e-05 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.950446+0000", 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100279335 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.950482+0000", 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6226000000000002e-05 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.115 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.953653+0000", 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0031705079999999998 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.953745+0000", 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1664999999999997e-05 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.116 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 72/31 ECSubWrite(tid=63, reqid=osd.11.0:63, at_version=72'32, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.837897+0000", 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.256326230999999, 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203598052, 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837897+0000", 2024-04-18T23:12:06.117 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837897+0000", 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837898+0000", 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8700000000000005e-07 2024-04-18T23:12:06.118 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837972+0000", 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4115000000000004e-05 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837995+0000", 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2589999999999999e-05 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.119 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838014+0000", 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8879e-05 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.938244+0000", 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100230344 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.938265+0000", 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1396e-05 2024-04-18T23:12:06.120 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041423+0000", 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10315798700000001 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041495+0000", 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1953999999999996e-05 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.121 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 72/31 ECSubWrite(tid=64, reqid=client.4568.0:562, at_version=72'33, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.838056+0000", 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 27.256167483999999, 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203492644, 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.122 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838056+0000", 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838056+0000", 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838057+0000", 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3399999999999997e-07 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.123 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838308+0000", 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025151500000000001 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838327+0000", 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8638999999999999e-05 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.124 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838339+0000", 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1950000000000001e-05 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.039312+0000", 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20097348100000001 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.039330+0000", 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7672e-05 2024-04-18T23:12:06.125 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041507+0000", 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021768360000000001 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041549+0000", 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1616999999999998e-05 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.126 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 78/31 ECSubWrite(tid=107, reqid=osd.7.0:107, at_version=78'86, trim_to=0'0, roll_forward_to=77'84))", 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.726020+0000", 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.368203211000001, 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10314021499999999, 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726020+0000", 2024-04-18T23:12:06.127 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726020+0000", 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726021+0000", 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7000000000000004e-07 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.128 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726077+0000", 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5779999999999998e-05 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726112+0000", 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4863999999999997e-05 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.129 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.726142+0000", 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9958e-05 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.826516+0000", 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10037412499999999 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.826550+0000", 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3797999999999997e-05 2024-04-18T23:12:06.130 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.829081+0000", 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025316090000000002 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.829160+0000", 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9311e-05 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.131 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 78/72 ECSubWrite(tid=175, reqid=osd.14.0:175, at_version=78'82, trim_to=0'0, roll_forward_to=78'80))", 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:45.028035+0000", 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.066188062999998, 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104985259, 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028035+0000", 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.132 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028035+0000", 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028037+0000", 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.576e-06 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.133 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028115+0000", 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8271999999999999e-05 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028149+0000", 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3943999999999999e-05 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.028173+0000", 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3875000000000001e-05 2024-04-18T23:12:06.134 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.130996+0000", 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102822806 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.131031+0000", 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5243999999999997e-05 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.132952+0000", 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019205369999999999 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.133021+0000", 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9004999999999994e-05 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 82/72 ECSubWrite(tid=194, reqid=client.4568.0:674, at_version=82'98, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.661023+0000", 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.433200593999999, 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30022358900000001, 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661023+0000", 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661023+0000", 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661024+0000", 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661197+0000", 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017270400000000001 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661218+0000", 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1543e-05 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661229+0000", 2024-04-18T23:12:06.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0539e-05 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.858998+0000", 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.197769312 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.859008+0000", 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0365e-05 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961168+0000", 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10215980500000001 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961246+0000", 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8355000000000005e-05 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 82/72 ECSubWrite(tid=195, reqid=client.4568.0:675, at_version=82'99, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.760788+0000", 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.333435765000001, 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20119051900000001, 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.760788+0000", 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.760788+0000", 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.760790+0000", 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1160000000000002e-06 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761152+0000", 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036268500000000001 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761191+0000", 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8374000000000002e-05 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761221+0000", 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0431999999999999e-05 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.859990+0000", 2024-04-18T23:12:06.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098768442999999997 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.860005+0000", 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5036e-05 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961955+0000", 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10195014400000001 2024-04-18T23:12:06.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961978+0000", 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3289e-05 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 82/72 ECSubWrite(tid=196, reqid=client.4568.0:676, at_version=82'100, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.762849+0000", 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.331374636, 2024-04-18T23:12:06.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199319412, 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762849+0000", 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762849+0000", 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762851+0000", 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.81e-06 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.763087+0000", 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236512 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.763123+0000", 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6171000000000003e-05 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.763151+0000", 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7212e-05 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961317+0000", 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19816698099999999 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961328+0000", 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0524e-05 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.962142+0000", 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081420799999999997 2024-04-18T23:12:06.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.962168+0000", 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5993999999999999e-05 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 84/31 ECSubWrite(tid=148, reqid=client.4568.0:750, at_version=84'124, trim_to=0'0, roll_forward_to=83'121))", 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.980774+0000", 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.113449620000001, 2024-04-18T23:12:06.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201355073, 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.980774+0000", 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.980774+0000", 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.980775+0000", 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5370000000000001e-06 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.981058+0000", 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028255700000000001 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.981081+0000", 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3535000000000001e-05 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.981103+0000", 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1265999999999999e-05 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.179112+0000", 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198009659 2024-04-18T23:12:06.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.179140+0000", 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8008999999999999e-05 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.182090+0000", 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029498990000000002 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.156 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.182129+0000", 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8611e-05 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=173, reqid=client.4568.0:906, at_version=98'150, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.679824+0000", 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4143993749999999, 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103090924, 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.157 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.679824+0000", 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.679824+0000", 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.679824+0000", 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7399999999999999e-07 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.680285+0000", 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00046092600000000001 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.680305+0000", 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9598000000000001e-05 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.680319+0000", 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3821e-05 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.780499+0000", 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100180407 2024-04-18T23:12:06.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.780517+0000", 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8326e-05 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.782819+0000", 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0023018219999999998 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.782915+0000", 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5749999999999996e-05 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=174, reqid=client.4568.0:907, at_version=98'151, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.683135+0000", 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.4110884530000001, 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30320944399999999, 2024-04-18T23:12:06.162 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683135+0000", 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683135+0000", 2024-04-18T23:12:06.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683136+0000", 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683285+0000", 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149352 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683304+0000", 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8508999999999998e-05 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783382+0000", 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10007812000000001 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783518+0000", 2024-04-18T23:12:06.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013595999999999999 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783549+0000", 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1309000000000001e-05 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986291+0000", 2024-04-18T23:12:06.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202741639 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986344+0000", 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3594999999999998e-05 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 98/31 ECSubWrite(tid=103, reqid=client.4568.0:905, at_version=98'65, trim_to=0'0, roll_forward_to=95'63))", 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.767160+0000", 2024-04-18T23:12:06.167 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.327063493, 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103178199, 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.767160+0000", 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.767160+0000", 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.767161+0000", 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.9999999999999997e-07 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.767380+0000", 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021926500000000001 2024-04-18T23:12:06.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.767411+0000", 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1192000000000002e-05 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.767432+0000", 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0628999999999998e-05 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.867694+0000", 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100262324 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.867713+0000", 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8649000000000001e-05 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.870300+0000", 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0025877769999999999 2024-04-18T23:12:06.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.870338+0000", 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7762999999999997e-05 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=175, reqid=client.4568.0:908, at_version=98'152, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.783421+0000", 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3108025510000001, 2024-04-18T23:12:06.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20296605300000001, 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783421+0000", 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783421+0000", 2024-04-18T23:12:06.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783422+0000", 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2699999999999998e-07 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783585+0000", 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163673 2024-04-18T23:12:06.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783611+0000", 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5644999999999999e-05 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783626+0000", 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4430000000000001e-05 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.784619+0000", 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099339299999999997 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.784645+0000", 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5896000000000001e-05 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986354+0000", 2024-04-18T23:12:06.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20170896799999999 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986387+0000", 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3120999999999997e-05 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=176, reqid=client.4568.0:909, at_version=98'153, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.783642+0000", 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3105813150000001, 2024-04-18T23:12:06.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20277439999999999, 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783642+0000", 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783642+0000", 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783643+0000", 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.89e-07 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783751+0000", 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108342 2024-04-18T23:12:06.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783760+0000", 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.414e-06 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783767+0000", 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.216e-06 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.785732+0000", 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019647110000000001 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.785759+0000", 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7064999999999999e-05 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986391+0000", 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20063127 2024-04-18T23:12:06.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986417+0000", 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5993e-05 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=177, reqid=client.4568.0:910, at_version=98'154, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.783778+0000", 2024-04-18T23:12:06.182 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.3104452150000001, 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203022498, 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783778+0000", 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783778+0000", 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783779+0000", 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.27e-07 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783797+0000", 2024-04-18T23:12:06.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8011000000000001e-05 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783804+0000", 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7049999999999999e-06 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783811+0000", 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2519999999999996e-06 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.985869+0000", 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20205835 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.985886+0000", 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7359000000000001e-05 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986768+0000", 2024-04-18T23:12:06.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088193500000000003 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986801+0000", 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2558999999999998e-05 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=178, reqid=client.4568.0:914, at_version=98'155, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.887097+0000", 2024-04-18T23:12:06.187 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.2071259599999999, 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104123337, 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.887097+0000", 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.887097+0000", 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.887100+0000", 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5399999999999998e-06 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.887348+0000", 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00024753699999999998 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.887380+0000", 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2042000000000003e-05 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.987459+0000", 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100079818 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.987543+0000", 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.3353e-05 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.987549+0000", 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4030000000000004e-06 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.991171+0000", 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0036219730000000001 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.991221+0000", 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.9670999999999999e-05 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 98/31 ECSubRead(tid=183, to_read={3:b7d0d2d3:::smithi14230322-10:head=0,647168,0}, subchunks={3:b7d0d2d3:::smithi14230322-10:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.988115+0000", 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 1.1061085559999999, 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20139596500000001, 2024-04-18T23:12:06.192 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988115+0000", 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988115+0000", 2024-04-18T23:12:06.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988115+0000", 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3799999999999998e-07 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988124+0000", 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.242e-06 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988137+0000", 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3468e-05 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:05.088254+0000", 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100116893 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:05.188513+0000", 2024-04-18T23:12:06.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100258602 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:05.189511+0000", 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099832200000000005 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.196 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:06.198 INFO:tasks.rados.rados.0.smithi142.stdout:367: finishing write tid 6 to smithi14230322-35 2024-04-18T23:12:06.198 INFO:tasks.rados.rados.0.smithi142.stdout:367: oid 35 updating version 0 to 179 2024-04-18T23:12:06.198 INFO:tasks.rados.rados.0.smithi142.stdout:367: oid 35 version 179 is already newer than 177 2024-04-18T23:12:06.199 INFO:tasks.rados.rados.0.smithi142.stdout:367: oid 35 version 179 is already newer than 178 2024-04-18T23:12:06.199 INFO:tasks.rados.rados.0.smithi142.stdout:367: oid 35 updating version 179 to 181 2024-04-18T23:12:06.199 INFO:tasks.rados.rados.0.smithi142.stdout:367: oid 35 version 181 is already newer than 180 2024-04-18T23:12:06.199 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 35 v 181 (ObjNum 134 snap 35 seq_num 134) dirty exists 2024-04-18T23:12:06.199 INFO:tasks.rados.rados.0.smithi142.stdout:367: left oid 35 (ObjNum 134 snap 35 seq_num 134) 2024-04-18T23:12:06.199 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:06.146+0000 7f9e99dca640 -1 snap_mapper.get_next_objects_to_trim::New Clone-Objects were added to Snap 23 after trimming was started 2024-04-18T23:12:06.199 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:06.194+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:06.200 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:12:06.220 INFO:tasks.rados.rados.0.smithi142.stdout:369: finishing copy_from to smithi14230322-49 2024-04-18T23:12:06.220 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 49 v 42 (ObjNum 120 snap 27 seq_num 120) dirty exists 2024-04-18T23:12:06.299 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:06.294+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:06.321 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.9s0 99/72 ECSubWriteReply(tid=254, last_complete=99'145, committed=1, applied=1))", 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.198360+0000", 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 0.12523886000000001, 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.12527913900000001, 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.198360+0000", 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.198360+0000", 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.198362+0000", 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.296e-06 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.198371+0000", 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1230000000000006e-06 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.198392+0000", 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1702e-05 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.198407+0000", 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4440000000000001e-05 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.324 INFO:teuthology.orchestra.run.smithi136.stdout: ], 2024-04-18T23:12:06.325 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 1 2024-04-18T23:12:06.325 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:06.337 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:12:06.399 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:06.394+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:06.459 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:12:06.459 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:12:06.460 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:06.474 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:12:06.501 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:06.499+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:06.571 INFO:tasks.rados.rados.0.smithi142.stdout:355: done (8 left) 2024-04-18T23:12:06.571 INFO:tasks.rados.rados.0.smithi142.stdout:357: done (7 left) 2024-04-18T23:12:06.571 INFO:tasks.rados.rados.0.smithi142.stdout:361: done (6 left) 2024-04-18T23:12:06.571 INFO:tasks.rados.rados.0.smithi142.stdout:362: done (5 left) 2024-04-18T23:12:06.571 INFO:tasks.rados.rados.0.smithi142.stdout:364: done (4 left) 2024-04-18T23:12:06.571 INFO:tasks.rados.rados.0.smithi142.stdout:367: done (3 left) 2024-04-18T23:12:06.571 INFO:tasks.rados.rados.0.smithi142.stdout:369: done (2 left) 2024-04-18T23:12:06.572 INFO:tasks.rados.rados.0.smithi142.stdout:371: done (1 left) 2024-04-18T23:12:06.572 INFO:tasks.rados.rados.0.smithi142.stdout:372: read oid 49 snap -1 2024-04-18T23:12:06.572 INFO:tasks.rados.rados.0.smithi142.stdout:372: expect (ObjNum 120 snap 27 seq_num 120) 2024-04-18T23:12:06.572 INFO:tasks.rados.rados.0.smithi142.stdout:373: copy_from oid 20 from oid 35 current snap is 36 2024-04-18T23:12:06.572 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 41 current snap is 36 2024-04-18T23:12:06.572 INFO:tasks.rados.rados.0.smithi142.stdout:374: seq_num 135 ranges {2228224=720896,2949120=622592,3571712=688128,4259840=245760} 2024-04-18T23:12:06.572 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-04-18T23:12:06.587 INFO:tasks.rados.rados.0.smithi142.stdout:374: writing smithi14230322-41 from 2228224 to 2949120 tid 1 2024-04-18T23:12:06.590 INFO:tasks.rados.rados.0.smithi142.stdout:374: writing smithi14230322-41 from 2949120 to 3571712 tid 2 2024-04-18T23:12:06.595 INFO:tasks.rados.rados.0.smithi142.stdout:374: writing smithi14230322-41 from 3571712 to 4259840 tid 3 2024-04-18T23:12:06.596 INFO:tasks.rados.rados.0.smithi142.stdout:374: writing smithi14230322-41 from 4259840 to 4505600 tid 4 2024-04-18T23:12:06.597 INFO:tasks.rados.rados.0.smithi142.stdout:375: read oid 34 snap -1 2024-04-18T23:12:06.597 INFO:tasks.rados.rados.0.smithi142.stdout:375: expect deleted 2024-04-18T23:12:06.597 INFO:tasks.rados.rados.0.smithi142.stdout:376: rmattr oid 31 current snap is 36 2024-04-18T23:12:06.597 INFO:tasks.rados.rados.0.smithi142.stdout:376: done (5 left) 2024-04-18T23:12:06.597 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 33 current snap is 36 2024-04-18T23:12:06.597 INFO:tasks.rados.rados.0.smithi142.stdout:377: seq_num 136 ranges {3293184=737280,4030464=770048,4800512=540672,5341184=245760} 2024-04-18T23:12:06.601 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:06.600+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:06.606 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:12:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:12:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:12:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:273 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:12:06.606 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.022983574000001, 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.781474428, 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 273 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:12:06.607 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581589+0000", 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581592+0000", 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999999e-06 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581611+0000", 2024-04-18T23:12:06.608 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8736000000000002e-05 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.581652+0000", 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1421999999999997e-05 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681775+0000", 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100122909 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.609 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681951+0000", 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000175358 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681971+0000", 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0795e-05 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.858775+0000", 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17680337099999999 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.610 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.858953+0000", 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017833000000000001 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.363063+0000", 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50410986700000004 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.611 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:274 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.922737464999997, 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.68252750799999995, 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 274 2024-04-18T23:12:06.612 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681835+0000", 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681837+0000", 2024-04-18T23:12:06.613 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.756e-06 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681852+0000", 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4566000000000001e-05 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.681881+0000", 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8651999999999999e-05 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.614 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.781982+0000", 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100101015 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782144+0000", 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000162347 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782164+0000", 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9888999999999999e-05 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.959687+0000", 2024-04-18T23:12:06.615 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.17752280100000001 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.959899+0000", 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021188899999999999 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364362+0000", 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40446359300000001 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.616 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:275 3.0s0 3:01c2b7c9:::smithi14230322-29:6 [read 0~475136,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.822553051, 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.78490300599999996, 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 275 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.617 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782019+0000", 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782021+0000", 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.497e-06 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.618 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782031+0000", 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8730000000000002e-06 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782049+0000", 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8016000000000002e-05 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782060+0000", 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1066e-05 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.619 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782264+0000", 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020450899999999999 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:06.782285+0000", 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1066000000000001e-05 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364492+0000", 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.58220711199999997 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.620 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.364799+0000", 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00030637600000000002 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:07.566922+0000", 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20212349099999999 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:341 3.9s0 3:906aad0a:::smithi14230322-27:head [append 0~0] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:12:06.621 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.724969+0000", 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.879603426999999, 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.60935175900000005, 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 341 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724969+0000", 2024-04-18T23:12:06.622 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724969+0000", 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724971+0000", 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4899999999999999e-06 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.623 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.724985+0000", 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.418e-05 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725014+0000", 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9016000000000001e-05 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725034+0000", 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9825000000000001e-05 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.624 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725162+0000", 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012797000000000001 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725175+0000", 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2255999999999999e-05 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828411+0000", 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103236438 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.625 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.828613+0000", 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00020252399999999999 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.830448+0000", 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00183472 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.932371+0000", 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101922922 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:06.626 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.334124+0000", 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40175268400000003 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.334321+0000", 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000196734 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:342 3.9s0 3:906aad0a:::smithi14230322-27:head [rollback 4] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.725072+0000", 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.879500120000003, 2024-04-18T23:12:06.627 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71010575600000003, 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 342 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725072+0000", 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.628 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725072+0000", 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725074+0000", 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.48e-06 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725081+0000", 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9959999999999996e-06 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.629 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725098+0000", 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7785999999999999e-05 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725112+0000", 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.403e-05 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725303+0000", 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019035999999999999 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.630 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725342+0000", 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8769000000000001e-05 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.928317+0000", 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20297505199999999 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.928568+0000", 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025112799999999998 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.631 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.929483+0000", 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000915371 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.932373+0000", 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.002890218 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.434969+0000", 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50259540999999996 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.632 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435178+0000", 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000209156 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:343 3.9s0 3:906aad0a:::smithi14230322-27:head [append 0~0] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.725142+0000", 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.879430036000002, 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.71044070299999995, 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.633 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 343 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725142+0000", 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725142+0000", 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.634 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725144+0000", 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.347e-06 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725169+0000", 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5683000000000001e-05 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725187+0000", 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8235000000000001e-05 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.635 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725203+0000", 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5933000000000001e-05 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725399+0000", 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019563999999999999 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.725408+0000", 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0999999999999993e-06 2024-04-18T23:12:06.636 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.930345+0000", 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.204937228 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.930628+0000", 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000282274 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.931715+0000", 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010869009999999999 2024-04-18T23:12:06.637 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.233078+0000", 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30136315400000002 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435450+0000", 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20237229500000001 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:13.435583+0000", 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013291299999999999 2024-04-18T23:12:06.638 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:421 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~557056] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.533981+0000", 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.070590975000002, 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81687969000000005, 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.639 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 421 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.533981+0000", 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.533981+0000", 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.640 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.533984+0000", 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.83e-06 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.534844+0000", 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00085963299999999995 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.534879+0000", 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5305999999999997e-05 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.534898+0000", 2024-04-18T23:12:06.641 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9340000000000001e-05 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.535053+0000", 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015493200000000001 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.535070+0000", 2024-04-18T23:12:06.642 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6682999999999999e-05 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.640162+0000", 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105092212 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.640474+0000", 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00031151200000000001 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.643 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643548+0000", 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003074181 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.652446+0000", 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0088978619999999994 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350532+0000", 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.69808661400000005 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.644 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350861+0000", 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000328585 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:422 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~671744] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.537766+0000", 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.066805725999998, 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81361762999999998, 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:06.645 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 422 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.537766+0000", 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.537766+0000", 2024-04-18T23:12:06.646 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.537769+0000", 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1059999999999998e-06 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538649+0000", 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088010300000000003 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538679+0000", 2024-04-18T23:12:06.647 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0263000000000001e-05 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538694+0000", 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4936000000000001e-05 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538836+0000", 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000141643 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.648 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538849+0000", 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3013000000000001e-05 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.646583+0000", 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.107734542 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.646882+0000", 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029901899999999998 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.649 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.648879+0000", 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019973460000000001 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.653818+0000", 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0049380880000000002 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.351099+0000", 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.69728165900000005 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.650 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.351384+0000", 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028491200000000001 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:423 3.0s0 3:063b0705:::smithi14230322-16:head [append 0~98304] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.538732+0000", 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.065840231999999, 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.91325199599999995, 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.651 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 423 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538732+0000", 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.652 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538732+0000", 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538733+0000", 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538808+0000", 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4623999999999996e-05 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.653 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538822+0000", 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.401e-05 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538832+0000", 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7229999999999999e-06 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538898+0000", 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6571999999999999e-05 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.654 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538907+0000", 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.003e-06 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.650070+0000", 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.111163027 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.650383+0000", 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000312844 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.655 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.651521+0000", 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001137817 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.654205+0000", 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026839479999999998 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.451402+0000", 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.79719673599999996 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.656 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.451984+0000", 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00058231199999999998 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:424 3.0s0 3:063b0705:::smithi14230322-16:head [setxattr _header (57) in=7b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.538853+0000", 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.065718941999997, 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.014298884, 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:06.657 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 424 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538853+0000", 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538853+0000", 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.658 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538854+0000", 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7300000000000004e-07 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538859+0000", 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6299999999999997e-06 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538869+0000", 2024-04-18T23:12:06.659 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0302999999999999e-05 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538881+0000", 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2196e-05 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538998+0000", 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000116821 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.539024+0000", 2024-04-18T23:12:06.660 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5392e-05 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.652483+0000", 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.113459882 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.652660+0000", 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000176522 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.653461+0000", 2024-04-18T23:12:06.661 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000801299 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.655473+0000", 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020117630000000002 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.552510+0000", 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.89703736599999995 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.662 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.553152+0000", 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00064173699999999997 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:425 3.0s0 3:063b0705:::smithi14230322-16:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.538897+0000", 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 45.065675315999997, 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.219395432, 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.663 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 425 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538897+0000", 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538897+0000", 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.664 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538898+0000", 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8599999999999997e-07 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538901+0000", 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6169999999999998e-06 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538912+0000", 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0412e-05 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.665 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.538920+0000", 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5569999999999996e-06 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.638222+0000", 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099301983999999996 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.638243+0000", 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0825000000000001e-05 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.666 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.654027+0000", 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.015784231999999999 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.654189+0000", 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000161686 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.654544+0000", 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.000355017 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.667 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.654740+0000", 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019602899999999999 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.758292+0000", 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10355228700000001 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=94, last_complete=56'16, committed=1, applied=1))", 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.752292+0000", 2024-04-18T23:12:06.668 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.852280211999997, 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497650382, 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752292+0000", 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752292+0000", 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.669 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752294+0000", 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.767e-06 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752304+0000", 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9520000000000006e-06 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.752329+0000", 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5743000000000001e-05 2024-04-18T23:12:06.670 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852428+0000", 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100098916 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.249913+0000", 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39748424700000001 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.249942+0000", 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9757e-05 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.671 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=95, last_complete=56'17, committed=1, applied=1))", 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.852480+0000", 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.752091718999999, 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49763560899999998, 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852480+0000", 2024-04-18T23:12:06.672 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852480+0000", 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852482+0000", 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0470000000000001e-06 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852489+0000", 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7100000000000007e-06 2024-04-18T23:12:06.673 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852510+0000", 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1109000000000001e-05 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852521+0000", 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.11e-05 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350081+0000", 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49755995800000002 2024-04-18T23:12:06.674 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350116+0000", 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4684999999999997e-05 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=96, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.752023166999997, 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49767800099999998, 2024-04-18T23:12:06.675 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.676 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852549+0000", 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1300000000000001e-07 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852553+0000", 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8999999999999999e-06 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852566+0000", 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3063e-05 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.677 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.852574+0000", 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5440000000000001e-06 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350198+0000", 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.497624179 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350227+0000", 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8901999999999999e-05 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.678 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=97, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.853074+0000", 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.751498464999997, 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49724826700000002, 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853074+0000", 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.679 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853074+0000", 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853075+0000", 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1999999999999999e-06 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853082+0000", 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1999999999999997e-06 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.680 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853100+0000", 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8357000000000001e-05 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.853111+0000", 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0389e-05 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350293+0000", 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49718215799999999 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.681 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.350322+0000", 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8963e-05 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=96, last_complete=56'18, committed=1, applied=1))", 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.917214+0000", 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.687358119000002, 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53479636500000005, 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.682 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917214+0000", 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917214+0000", 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917215+0000", 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1999999999999999e-07 2024-04-18T23:12:06.683 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917220+0000", 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.7299999999999996e-06 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917235+0000", 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5273000000000001e-05 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917244+0000", 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0359999999999995e-06 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.684 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.451137+0000", 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53389361899999999 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.452010+0000", 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00087298699999999998 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.0s0 56/49 ECSubWriteReply(tid=97, last_complete=56'19, committed=1, applied=1))", 2024-04-18T23:12:06.685 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.917619+0000", 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.686952980000001, 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.636734154, 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917619+0000", 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917619+0000", 2024-04-18T23:12:06.686 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917620+0000", 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9399999999999993e-07 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917627+0000", 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.8360000000000003e-06 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.917645+0000", 2024-04-18T23:12:06.687 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7566000000000001e-05 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.017700+0000", 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100055057 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.552241+0000", 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.53454112899999995 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:22.554353+0000", 2024-04-18T23:12:06.688 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021125720000000001 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:676 3.9s0 3:9e926f5e:::smithi14230322-42:head [append 0~409600] snapc 1a=[19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e82)", 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.599192+0000", 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 18.005379938000001, 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.56959064999999998, 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.689 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 676 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599192+0000", 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599192+0000", 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.690 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599194+0000", 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2170000000000001e-06 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599700+0000", 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00050507899999999997 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.599731+0000", 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1477999999999999e-05 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.691 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699847+0000", 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100115674 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761918+0000", 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.062071227 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762022+0000", 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00010433999999999999 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.692 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762761+0000", 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00073854300000000003 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.964670+0000", 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201908694 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.168699+0000", 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20402904999999999 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.693 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.168783+0000", 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4347999999999999e-05 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:678 3.9s0 3:9e926f5e:::smithi14230322-42:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.699915+0000", 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.904657182000001, 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.67171341299999998, 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:06.694 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 678 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699915+0000", 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699915+0000", 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.695 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699915+0000", 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9000000000000002e-07 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699918+0000", 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0990000000000001e-06 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699923+0000", 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3510000000000001e-06 2024-04-18T23:12:06.696 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.699927+0000", 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2339999999999996e-06 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.864219+0000", 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.16429192200000001 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.864302+0000", 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.2857999999999998e-05 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.697 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.169513+0000", 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30521089099999998 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.169614+0000", 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000101147 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.371628+0000", 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202014521 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.698 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:763 3.9s0 3:9e439d70:::smithi14230322-3:head [append 0~0] snapc 1b=[1b,19,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e84)", 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.782684+0000", 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.821888477, 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.49707129799999999, 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 763 2024-04-18T23:12:06.699 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.782684+0000", 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.782684+0000", 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.782685+0000", 2024-04-18T23:12:06.700 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5370000000000001e-06 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.782693+0000", 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2799999999999998e-06 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.782713+0000", 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0882000000000001e-05 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.882806+0000", 2024-04-18T23:12:06.701 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100092959 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.882973+0000", 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016678 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.883281+0000", 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000308299 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.884664+0000", 2024-04-18T23:12:06.702 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013824320000000001 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.986881+0000", 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10221706699999999 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.279627+0000", 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29274583999999998 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.279755+0000", 2024-04-18T23:12:06.703 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012822199999999999 2024-04-18T23:12:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:06.704 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:06.704 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:06.706 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:377: writing smithi14230322-33 from 3293184 to 4030464 tid 1 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:377: writing smithi14230322-33 from 4030464 to 4800512 tid 2 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:377: writing smithi14230322-33 from 4800512 to 5341184 tid 3 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:377: writing smithi14230322-33 from 5341184 to 5586944 tid 4 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:378: read oid 31 snap -1 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:378: expect (ObjNum 115 snap 26 seq_num 115) 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:379: copy_from oid 47 from oid 23 current snap is 36 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:380: read oid 16 snap -1 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:380: expect (ObjNum 120 snap 27 seq_num 120) 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:381: read oid 7 snap 26 2024-04-18T23:12:06.707 INFO:tasks.rados.rados.0.smithi142.stdout:381: expect deleted 2024-04-18T23:12:06.708 INFO:tasks.rados.rados.0.smithi142.stdout:382: read oid 26 snap -1 2024-04-18T23:12:06.708 INFO:tasks.rados.rados.0.smithi142.stdout:382: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:12:06.708 INFO:tasks.rados.rados.0.smithi142.stdout:383: snap_create 2024-04-18T23:12:06.708 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:12:06.703+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:12:06.719 INFO:tasks.rados.rados.0.smithi142.stdout:370: finishing copy_from racing read to smithi14230322-5 2024-04-18T23:12:06.760 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:12:06.806 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:06.802+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:06.823 INFO:tasks.rados.rados.0.smithi142.stdout:370: finishing copy_from to smithi14230322-5 2024-04-18T23:12:06.823 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 5 v 182 (ObjNum 119 snap 26 seq_num 119) dirty exists 2024-04-18T23:12:06.840 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:06.840 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:12:06.840 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.as0 100/31 ECSubWriteReply(tid=55, last_complete=100'30, committed=1, applied=1))", 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.827023+0000", 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 0.014901532, 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.014937908, 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "queued for pg", 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.827023+0000", 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:06.841 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:06.842 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.827023+0000", 2024-04-18T23:12:06.842 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:06.842 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:06.842 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:06.842 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:06.842 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.827024+0000", 2024-04-18T23:12:06.842 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2699999999999999e-06 2024-04-18T23:12:06.842 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.827032+0000", 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1300000000000001e-06 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.827055+0000", 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2308999999999999e-05 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.827070+0000", 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5690000000000001e-05 2024-04-18T23:12:06.843 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:06.844 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:06.844 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:06.844 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:06.844 INFO:teuthology.orchestra.run.smithi142.stdout: ], 2024-04-18T23:12:06.844 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 1 2024-04-18T23:12:06.844 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:06.854 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:12:06.904 INFO:tasks.rados.rados.0.smithi142.stdout:382: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:12:06.907 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:06.908+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:06.924 INFO:tasks.rados.rados.0.smithi142.stdout:373: finishing copy_from racing read to smithi14230322-20 2024-04-18T23:12:06.930 INFO:tasks.rados.rados.0.smithi142.stdout:374: finishing write tid 1 to smithi14230322-41 2024-04-18T23:12:06.930 INFO:tasks.rados.rados.0.smithi142.stdout:374: finishing write tid 2 to smithi14230322-41 2024-04-18T23:12:06.990 INFO:tasks.rados.rados.0.smithi142.stdout:377: finishing write tid 1 to smithi14230322-33 2024-04-18T23:12:07.008 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:07.006+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:07.032 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:07.032 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:12:07.032 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:12:07.032 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:12:07.032 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:07.033 INFO:tasks.rados.rados.0.smithi142.stdout:374: finishing write tid 3 to smithi14230322-41 2024-04-18T23:12:07.034 INFO:tasks.rados.rados.0.smithi142.stdout:374: finishing write tid 4 to smithi14230322-41 2024-04-18T23:12:07.040 INFO:tasks.rados.rados.0.smithi142.stdout:378: expect (ObjNum 115 snap 26 seq_num 115) 2024-04-18T23:12:07.050 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:12:07.061 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:12:07.061 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:12:07.090 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:12:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:12:07.109 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:07.106+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:07.118 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:12:07.118 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:12:07.141 INFO:tasks.rados.rados.0.smithi142.stdout:374: finishing write tid 5 to smithi14230322-41 2024-04-18T23:12:07.141 INFO:tasks.rados.rados.0.smithi142.stdout:380: expect (ObjNum 120 snap 27 seq_num 120) 2024-04-18T23:12:07.210 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:07.206+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:07.215 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:12:07.215 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:12:07.223 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:07.223 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:07.237 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:12:07.284 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:12:07.284 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:12:07.311 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:07.310+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:07.318 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:12:07.318 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:12:07.351 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:12:07.351 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:12:07.382 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:12:07.382 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:12:07.399 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:07.399 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:07.410 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:12:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:12:07.413 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:12:07.413 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:07.412+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:07.428 INFO:tasks.rados.rados.0.smithi142.stdout:377: finishing write tid 2 to smithi14230322-33 2024-04-18T23:12:07.428 INFO:tasks.rados.rados.0.smithi142.stdout:377: finishing write tid 3 to smithi14230322-33 2024-04-18T23:12:07.428 INFO:tasks.rados.rados.0.smithi142.stdout:377: finishing write tid 4 to smithi14230322-33 2024-04-18T23:12:07.428 INFO:tasks.rados.rados.0.smithi142.stdout:377: finishing write tid 5 to smithi14230322-33 2024-04-18T23:12:07.429 INFO:tasks.rados.rados.0.smithi142.stdout:373: finishing copy_from to smithi14230322-20 2024-04-18T23:12:07.429 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 20 v 181 (ObjNum 134 snap 35 seq_num 134) dirty exists 2024-04-18T23:12:07.429 INFO:tasks.rados.rados.0.smithi142.stdout:379: finishing copy_from racing read to smithi14230322-47 2024-04-18T23:12:07.429 INFO:tasks.rados.rados.0.smithi142.stdout:372: expect (ObjNum 120 snap 27 seq_num 120) 2024-04-18T23:12:07.439 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:12:07.439 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:12:07.468 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:12:07.468 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:12:07.491 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:12:07.491 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:12:07.513 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:12:07.513 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:12:07.514 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:07.510+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:07.539 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:12:07.539 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:12:07.540 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:07.540 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:07.554 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:12:07.560 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:12:07.560 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:12:07.581 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:12:07.581 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:12:07.615 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:07.610+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:07.714 INFO:tasks.rados.rados.0.smithi142.stdout:374: finishing write tid 6 to smithi14230322-41 2024-04-18T23:12:07.714 INFO:tasks.rados.rados.0.smithi142.stdout:374: oid 41 updating version 0 to 165 2024-04-18T23:12:07.714 INFO:tasks.rados.rados.0.smithi142.stdout:374: oid 41 version 165 is already newer than 163 2024-04-18T23:12:07.714 INFO:tasks.rados.rados.0.smithi142.stdout:374: oid 41 updating version 165 to 167 2024-04-18T23:12:07.714 INFO:tasks.rados.rados.0.smithi142.stdout:374: oid 41 version 167 is already newer than 166 2024-04-18T23:12:07.714 INFO:tasks.rados.rados.0.smithi142.stdout:374: oid 41 version 167 is already newer than 164 2024-04-18T23:12:07.715 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 41 v 167 (ObjNum 135 snap 36 seq_num 135) dirty exists 2024-04-18T23:12:07.715 INFO:tasks.rados.rados.0.smithi142.stdout:374: left oid 41 (ObjNum 135 snap 36 seq_num 135) 2024-04-18T23:12:07.715 INFO:tasks.rados.rados.0.smithi142.stdout:377: finishing write tid 6 to smithi14230322-33 2024-04-18T23:12:07.715 INFO:tasks.rados.rados.0.smithi142.stdout:377: oid 33 updating version 0 to 129 2024-04-18T23:12:07.715 INFO:tasks.rados.rados.0.smithi142.stdout:377: oid 33 version 129 is already newer than 125 2024-04-18T23:12:07.715 INFO:tasks.rados.rados.0.smithi142.stdout:377: oid 33 version 129 is already newer than 127 2024-04-18T23:12:07.715 INFO:tasks.rados.rados.0.smithi142.stdout:377: oid 33 version 129 is already newer than 128 2024-04-18T23:12:07.715 INFO:tasks.rados.rados.0.smithi142.stdout:377: oid 33 version 129 is already newer than 126 2024-04-18T23:12:07.716 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 33 v 129 (ObjNum 136 snap 36 seq_num 136) dirty exists 2024-04-18T23:12:07.716 INFO:tasks.rados.rados.0.smithi142.stdout:377: left oid 33 (ObjNum 136 snap 36 seq_num 136) 2024-04-18T23:12:07.716 INFO:tasks.rados.rados.0.smithi142.stdout:379: finishing copy_from to smithi14230322-47 2024-04-18T23:12:07.716 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 47 v 161 (ObjNum 123 snap 31 seq_num 123) dirty exists 2024-04-18T23:12:07.716 INFO:tasks.rados.rados.0.smithi142.stdout:370: done (11 left) 2024-04-18T23:12:07.716 INFO:tasks.rados.rados.0.smithi142.stdout:372: done (10 left) 2024-04-18T23:12:07.716 INFO:tasks.rados.rados.0.smithi142.stdout:373: done (9 left) 2024-04-18T23:12:07.716 INFO:tasks.rados.rados.0.smithi142.stdout:374: done (8 left) 2024-04-18T23:12:07.717 INFO:tasks.rados.rados.0.smithi142.stdout:375: done (7 left) 2024-04-18T23:12:07.717 INFO:tasks.rados.rados.0.smithi142.stdout:377: done (6 left) 2024-04-18T23:12:07.717 INFO:tasks.rados.rados.0.smithi142.stdout:378: done (5 left) 2024-04-18T23:12:07.717 INFO:tasks.rados.rados.0.smithi142.stdout:379: done (4 left) 2024-04-18T23:12:07.717 INFO:tasks.rados.rados.0.smithi142.stdout:380: done (3 left) 2024-04-18T23:12:07.717 INFO:tasks.rados.rados.0.smithi142.stdout:381: done (2 left) 2024-04-18T23:12:07.717 INFO:tasks.rados.rados.0.smithi142.stdout:382: done (1 left) 2024-04-18T23:12:07.718 INFO:tasks.rados.rados.0.smithi142.stdout:383: done (0 left) 2024-04-18T23:12:07.718 INFO:tasks.rados.rados.0.smithi142.stdout:384: rollback oid 41 current snap is 37 2024-04-18T23:12:07.718 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 41 to 26 2024-04-18T23:12:07.718 INFO:tasks.rados.rados.0.smithi142.stdout:385: rollback oid 18 current snap is 37 2024-04-18T23:12:07.718 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 18 to 27 2024-04-18T23:12:07.718 INFO:tasks.rados.rados.0.smithi142.stdout:386: rmattr oid 23 current snap is 37 2024-04-18T23:12:07.719 INFO:tasks.rados.rados.0.smithi142.stdout:386: done (2 left) 2024-04-18T23:12:07.719 INFO:tasks.rados.rados.0.smithi142.stdout:387: copy_from oid 43 from oid 10 current snap is 37 2024-04-18T23:12:07.719 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 19 current snap is 37 2024-04-18T23:12:07.719 INFO:tasks.rados.rados.0.smithi142.stdout:388: seq_num 137 ranges {2588672=753664,3342336=786432,4128768=655360,4784128=114688} 2024-04-18T23:12:07.721 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:07.720+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:07.728 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:07.728 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:07.742 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:12:07.746 INFO:tasks.rados.rados.0.smithi142.stdout:388: writing smithi14230322-19 from 2588672 to 3342336 tid 1 2024-04-18T23:12:07.755 INFO:tasks.rados.rados.0.smithi142.stdout:388: writing smithi14230322-19 from 3342336 to 4128768 tid 2 2024-04-18T23:12:07.761 INFO:tasks.rados.rados.0.smithi142.stdout:388: writing smithi14230322-19 from 4128768 to 4784128 tid 3 2024-04-18T23:12:07.763 INFO:tasks.rados.rados.0.smithi142.stdout:388: writing smithi14230322-19 from 4784128 to 4898816 tid 4 2024-04-18T23:12:07.763 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 13 current snap is 37 2024-04-18T23:12:07.763 INFO:tasks.rados.rados.0.smithi142.stdout:389: seq_num 138 ranges {4669440=770048,5439488=540672,5980160=753664,6733824=262144} 2024-04-18T23:12:07.803 INFO:tasks.rados.rados.0.smithi142.stdout:389: writing smithi14230322-13 from 4669440 to 5439488 tid 1 2024-04-18T23:12:07.806 INFO:tasks.rados.rados.0.smithi142.stdout:389: writing smithi14230322-13 from 5439488 to 5980160 tid 2 2024-04-18T23:12:07.810 INFO:tasks.rados.rados.0.smithi142.stdout:389: writing smithi14230322-13 from 5980160 to 6733824 tid 3 2024-04-18T23:12:07.812 INFO:tasks.rados.rados.0.smithi142.stdout:389: writing smithi14230322-13 from 6733824 to 6995968 tid 4 2024-04-18T23:12:07.812 INFO:tasks.rados.rados.0.smithi142.stdout:390: setattr oid 26 current snap is 37 2024-04-18T23:12:07.813 INFO:tasks.rados.rados.0.smithi142.stdout:391: snap_create 2024-04-18T23:12:07.821 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:07.819+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:07.825 INFO:tasks.rados.rados.0.smithi142.stdout:384: finishing rollback tid 0 to smithi14230322-41 2024-04-18T23:12:07.862 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:07.862 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:07.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:12:07.922 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:07.918+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:07.925 INFO:tasks.rados.rados.0.smithi142.stdout:385: finishing rollback tid 0 to smithi14230322-18 2024-04-18T23:12:07.926 INFO:tasks.rados.rados.0.smithi142.stdout:385: finishing rollback tid 1 to smithi14230322-18 2024-04-18T23:12:08.022 INFO:tasks.rados.rados.0.smithi142.stdout:389: finishing write tid 1 to smithi14230322-13 2024-04-18T23:12:08.023 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:08.023+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:08.026 INFO:tasks.rados.rados.0.smithi142.stdout:385: finishing rollback tid 2 to smithi14230322-18 2024-04-18T23:12:08.026 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 18 v 121 (ObjNum 121 snap 27 seq_num 121) dirty exists 2024-04-18T23:12:08.028 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 26 v 147 (ObjNum 161 snap 0 seq_num 1) dirty exists 2024-04-18T23:12:08.035 INFO:tasks.rados.rados.0.smithi142.stdout:387: finishing copy_from racing read to smithi14230322-43 2024-04-18T23:12:08.042 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:08.042 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:08.056 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:12:08.057 INFO:tasks.rados.rados.0.smithi142.stdout:388: finishing write tid 1 to smithi14230322-19 2024-04-18T23:12:08.072 INFO:tasks.rados.rados.0.smithi142.stdout:388: finishing write tid 2 to smithi14230322-19 2024-04-18T23:12:08.123 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:08.124+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:08.127 INFO:tasks.rados.rados.0.smithi142.stdout:384: finishing rollback tid 1 to smithi14230322-41 2024-04-18T23:12:08.127 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 41 v 170 (ObjNum 10 snap 0 seq_num 304) dirty dne 2024-04-18T23:12:08.173 INFO:tasks.rados.rados.0.smithi142.stdout:388: finishing write tid 3 to smithi14230322-19 2024-04-18T23:12:08.176 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:08.176 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:08.190 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:12:08.222 INFO:tasks.rados.rados.0.smithi142.stdout:388: finishing write tid 4 to smithi14230322-19 2024-04-18T23:12:08.223 INFO:tasks.rados.rados.0.smithi142.stdout:389: finishing write tid 2 to smithi14230322-13 2024-04-18T23:12:08.224 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:08.222+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:08.226 INFO:tasks.rados.rados.0.smithi142.stdout:387: finishing copy_from to smithi14230322-43 2024-04-18T23:12:08.226 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 43 v 104 (ObjNum 107 snap 25 seq_num 107) dirty exists 2024-04-18T23:12:08.322 INFO:tasks.rados.rados.0.smithi142.stdout:388: finishing write tid 5 to smithi14230322-19 2024-04-18T23:12:08.324 INFO:tasks.rados.rados.0.smithi142.stdout:389: finishing write tid 3 to smithi14230322-13 2024-04-18T23:12:08.325 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:08.323+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:08.325 INFO:tasks.rados.rados.0.smithi142.stdout:389: finishing write tid 4 to smithi14230322-13 2024-04-18T23:12:08.349 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:08.349 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:08.363 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-18T23:12:08.427 INFO:tasks.rados.rados.0.smithi142.stdout:388: finishing write tid 6 to smithi14230322-19 2024-04-18T23:12:08.427 INFO:tasks.rados.rados.0.smithi142.stdout:388: oid 19 updating version 0 to 106 2024-04-18T23:12:08.427 INFO:tasks.rados.rados.0.smithi142.stdout:388: oid 19 updating version 106 to 109 2024-04-18T23:12:08.427 INFO:tasks.rados.rados.0.smithi142.stdout:388: oid 19 version 109 is already newer than 107 2024-04-18T23:12:08.427 INFO:tasks.rados.rados.0.smithi142.stdout:388: oid 19 version 109 is already newer than 108 2024-04-18T23:12:08.427 INFO:tasks.rados.rados.0.smithi142.stdout:388: oid 19 version 109 is already newer than 105 2024-04-18T23:12:08.427 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 19 v 109 (ObjNum 137 snap 37 seq_num 137) dirty exists 2024-04-18T23:12:08.427 INFO:tasks.rados.rados.0.smithi142.stdout:388: left oid 19 (ObjNum 137 snap 37 seq_num 137) 2024-04-18T23:12:08.428 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:08.427+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:08.516 INFO:tasks.rados.rados.0.smithi142.stdout:384: done (6 left) 2024-04-18T23:12:08.516 INFO:tasks.rados.rados.0.smithi142.stdout:385: done (5 left) 2024-04-18T23:12:08.516 INFO:tasks.rados.rados.0.smithi142.stdout:387: done (4 left) 2024-04-18T23:12:08.516 INFO:tasks.rados.rados.0.smithi142.stdout:388: done (3 left) 2024-04-18T23:12:08.516 INFO:tasks.rados.rados.0.smithi142.stdout:390: done (2 left) 2024-04-18T23:12:08.516 INFO:tasks.rados.rados.0.smithi142.stdout:391: done (1 left) 2024-04-18T23:12:08.517 INFO:tasks.rados.rados.0.smithi142.stdout:392: copy_from oid 17 from oid 20 current snap is 38 2024-04-18T23:12:08.517 INFO:tasks.rados.rados.0.smithi142.stdout:393: snap_remove snap 35 2024-04-18T23:12:08.527 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:08.528 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:08.528 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:08.528+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:08.542 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-18T23:12:08.599 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T23:12:08.628 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:08.626+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:08.711 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:08.712 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:08.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:12:08.728 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:08.727+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:08.829 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:08.826+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:08.853 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:08.853 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:08.867 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:12:08.930 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:12:08.926+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:12:08.988 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:08.988 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:09.002 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:12:09.030 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:09.026+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:09.132 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:12:09.130+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:12:09.165 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:09.166 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:09.179 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:12:09.233 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:09.232+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:09.307 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-18T23:12:09.307 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-18T23:12:09.308 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:12:09.334 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:09.330+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:09.345 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:09.345 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:09.360 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:12:09.434 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:09.430+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:09.518 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:393: done (2 left) 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:394: rollback oid 32 current snap is 38 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:rollback oid 32 to 31 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:389: finishing write tid 5 to smithi14230322-13 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:389: finishing write tid 6 to smithi14230322-13 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:389: oid 13 updating version 0 to 88 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:389: oid 13 version 88 is already newer than 87 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:389: oid 13 version 88 is already newer than 86 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:389: oid 13 updating version 88 to 89 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:389: oid 13 updating version 89 to 90 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 13 v 90 (ObjNum 138 snap 37 seq_num 138) dirty exists 2024-04-18T23:12:09.520 INFO:tasks.rados.rados.0.smithi142.stdout:389: left oid 13 (ObjNum 138 snap 37 seq_num 138) 2024-04-18T23:12:09.521 INFO:tasks.rados.rados.0.smithi142.stdout:392: finishing copy_from racing read to smithi14230322-17 2024-04-18T23:12:09.521 INFO:tasks.rados.rados.0.smithi142.stdout:392: finishing copy_from to smithi14230322-17 2024-04-18T23:12:09.521 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 17 v 181 (ObjNum 134 snap 35 seq_num 134) dirty exists 2024-04-18T23:12:09.521 INFO:tasks.rados.rados.0.smithi142.stdout:389: done (2 left) 2024-04-18T23:12:09.521 INFO:tasks.rados.rados.0.smithi142.stdout:392: done (1 left) 2024-04-18T23:12:09.521 INFO:tasks.rados.rados.0.smithi142.stdout:395: read oid 28 snap -1 2024-04-18T23:12:09.521 INFO:tasks.rados.rados.0.smithi142.stdout:395: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:12:09.521 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 21 current snap is 38 2024-04-18T23:12:09.521 INFO:tasks.rados.rados.0.smithi142.stdout:396: seq_num 139 ranges {3506176=786432,4292608=688128,4980736=475136,5455872=393216} 2024-04-18T23:12:09.525 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:09.525 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:09.536 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:12:09.536+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:12:09.542 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:12:09.550 INFO:tasks.rados.rados.0.smithi142.stdout:396: writing smithi14230322-21 from 3506176 to 4292608 tid 1 2024-04-18T23:12:09.555 INFO:tasks.rados.rados.0.smithi142.stdout:396: writing smithi14230322-21 from 4292608 to 4980736 tid 2 2024-04-18T23:12:09.559 INFO:tasks.rados.rados.0.smithi142.stdout:396: writing smithi14230322-21 from 4980736 to 5455872 tid 3 2024-04-18T23:12:09.561 INFO:tasks.rados.rados.0.smithi142.stdout:396: writing smithi14230322-21 from 5455872 to 5849088 tid 4 2024-04-18T23:12:09.562 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 4 current snap is 38 2024-04-18T23:12:09.562 INFO:tasks.rados.rados.0.smithi142.stdout:397: seq_num 140 ranges {2048000=802816,2850816=425984,3276800=720896,3997696=409600} 2024-04-18T23:12:09.582 INFO:tasks.rados.rados.0.smithi142.stdout:397: writing smithi14230322-4 from 2048000 to 2850816 tid 1 2024-04-18T23:12:09.585 INFO:tasks.rados.rados.0.smithi142.stdout:397: writing smithi14230322-4 from 2850816 to 3276800 tid 2 2024-04-18T23:12:09.589 INFO:tasks.rados.rados.0.smithi142.stdout:397: writing smithi14230322-4 from 3276800 to 3997696 tid 3 2024-04-18T23:12:09.592 INFO:tasks.rados.rados.0.smithi142.stdout:397: writing smithi14230322-4 from 3997696 to 4407296 tid 4 2024-04-18T23:12:09.592 INFO:tasks.rados.rados.0.smithi142.stdout:398: delete oid 9 current snap is 38 2024-04-18T23:12:09.599 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:12:09.600 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:12:09.600 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:12:09.623 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":132,"stamp":"2024-04-18T23:12:09.515174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136512032,"num_objects":131,"num_object_clones":79,"num_object_copies":907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":5,"num_read":584,"num_read_kb":146751,"num_write":1044,"num_write_kb":156388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1588,"ondisk_log_size":1588,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":735808,"kb_used_data":247828,"kb_used_omap":35,"kb_used_meta":487708,"kb_avail":1509213632,"statfs":{"total":1546188226560,"available":1545434759168,"internally_reserved":0,"allocated":253775872,"data_stored":369659994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":499413633},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,5,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":34750464,"num_objects":21,"num_object_clones":21,"num_object_copies":147,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":38,"num_read_kb":25663,"num_write":94,"num_write_kb":33936,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.950111"},"pg_stats":[{"pgid":"3.f","version":"80'19","reported_seq":338,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.583831+0000","last_change":"2024-04-18T23:12:05.584453+0000","last_active":"2024-04-18T23:12:05.583831+0000","last_peered":"2024-04-18T23:12:05.583831+0000","last_clean":"2024-04-18T23:12:05.583831+0000","last_became_active":"2024-04-18T23:10:54.696778+0000","last_became_peered":"2024-04-18T23:10:54.696778+0000","last_unstale":"2024-04-18T23:12:05.583831+0000","last_undegraded":"2024-04-18T23:12:05.583831+0000","last_fullsized":"2024-04-18T23:12:05.583831+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096931999999999997,"stat_sum":{"num_bytes":1933312,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19110,"num_write":6,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.e","version":"87'84","reported_seq":402,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.652030+0000","last_change":"2024-04-18T23:12:05.652610+0000","last_active":"2024-04-18T23:12:05.652030+0000","last_peered":"2024-04-18T23:12:05.652030+0000","last_clean":"2024-04-18T23:12:05.652030+0000","last_became_active":"2024-04-18T23:10:54.918517+0000","last_became_peered":"2024-04-18T23:10:54.918517+0000","last_unstale":"2024-04-18T23:12:05.652030+0000","last_undegraded":"2024-04-18T23:12:05.652030+0000","last_fullsized":"2024-04-18T23:12:05.652030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'84","last_scrub_stamp":"2024-04-18T23:12:01.429717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.429717+0000","objects_scrubbed":6,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:06.684408+0000","scrub_duration":815,"objects_trimmed":0,"snaptrim_duration":0.00088648799999999995,"stat_sum":{"num_bytes":6995968,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":16277,"num_write":54,"num_write_kb":8016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.d","version":"100'167","reported_seq":499,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:07.085218+0000","last_change":"2024-04-18T23:12:06.088542+0000","last_active":"2024-04-18T23:12:07.085218+0000","last_peered":"2024-04-18T23:12:07.085218+0000","last_clean":"2024-04-18T23:12:07.085218+0000","last_became_active":"2024-04-18T23:10:54.712663+0000","last_became_peered":"2024-04-18T23:10:54.712663+0000","last_unstale":"2024-04-18T23:12:07.085218+0000","last_undegraded":"2024-04-18T23:12:07.085218+0000","last_fullsized":"2024-04-18T23:12:07.085218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'148","last_scrub_stamp":"2024-04-18T23:12:03.759433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:03.759433+0000","objects_scrubbed":13,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:24.948937+0000","scrub_duration":319,"objects_trimmed":0,"snaptrim_duration":0.31019849700000002,"stat_sum":{"num_bytes":18448384,"num_objects":15,"num_object_clones":10,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":58,"num_read_kb":12309,"num_write":118,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.c","version":"99'67","reported_seq":380,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.788916+0000","last_change":"2024-04-18T23:12:05.790158+0000","last_active":"2024-04-18T23:12:05.788916+0000","last_peered":"2024-04-18T23:12:05.788916+0000","last_clean":"2024-04-18T23:12:05.788916+0000","last_became_active":"2024-04-18T23:10:54.717053+0000","last_became_peered":"2024-04-18T23:10:54.717053+0000","last_unstale":"2024-04-18T23:12:05.788916+0000","last_undegraded":"2024-04-18T23:12:05.788916+0000","last_fullsized":"2024-04-18T23:12:05.788916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:15.192684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.106631478,"stat_sum":{"num_bytes":9977856,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":22,"num_read_kb":8759,"num_write":42,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.b","version":"99'185","reported_seq":504,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.355644+0000","last_change":"2024-04-18T23:12:06.360059+0000","last_active":"2024-04-18T23:12:06.355644+0000","last_peered":"2024-04-18T23:12:06.355644+0000","last_clean":"2024-04-18T23:12:06.355644+0000","last_became_active":"2024-04-18T23:10:54.808152+0000","last_became_peered":"2024-04-18T23:10:54.808152+0000","last_unstale":"2024-04-18T23:12:06.355644+0000","last_undegraded":"2024-04-18T23:12:06.355644+0000","last_fullsized":"2024-04-18T23:12:06.355644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'175","last_scrub_stamp":"2024-04-18T23:12:03.002648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:03.002648+0000","objects_scrubbed":14,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:12.029331+0000","scrub_duration":819,"objects_trimmed":0,"snaptrim_duration":0.52115570700000002,"stat_sum":{"num_bytes":26034176,"num_objects":16,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":41,"num_read_kb":26756,"num_write":83,"num_write_kb":25696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.a","version":"99'28","reported_seq":357,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.683002+0000","last_change":"2024-04-18T23:12:05.684609+0000","last_active":"2024-04-18T23:12:05.683002+0000","last_peered":"2024-04-18T23:12:05.683002+0000","last_clean":"2024-04-18T23:12:05.683002+0000","last_became_active":"2024-04-18T23:10:54.764999+0000","last_became_peered":"2024-04-18T23:10:54.764999+0000","last_unstale":"2024-04-18T23:12:05.683002+0000","last_undegraded":"2024-04-18T23:12:05.683002+0000","last_fullsized":"2024-04-18T23:12:05.683002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.106914625,"stat_sum":{"num_bytes":720896,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":28,"num_read_kb":4232,"num_write":7,"num_write_kb":624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.9","version":"99'143","reported_seq":506,"reported_epoch":99,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:12:06.093010+0000","last_change":"2024-04-18T23:12:05.684570+0000","last_active":"2024-04-18T23:12:06.093010+0000","last_peered":"2024-04-18T23:12:06.093010+0000","last_clean":"2024-04-18T23:12:06.093010+0000","last_became_active":"2024-04-18T23:11:39.726824+0000","last_became_peered":"2024-04-18T23:11:39.726824+0000","last_unstale":"2024-04-18T23:12:06.093010+0000","last_undegraded":"2024-04-18T23:12:06.093010+0000","last_fullsized":"2024-04-18T23:12:06.093010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'124","last_scrub_stamp":"2024-04-18T23:11:59.890703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:11:59.890703+0000","objects_scrubbed":13,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:16.200845+0000","scrub_duration":619,"objects_trimmed":0,"snaptrim_duration":0.00090971900000000002,"stat_sum":{"num_bytes":5767168,"num_objects":17,"num_object_clones":11,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":56,"num_read_kb":10769,"num_write":108,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2867200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.8","version":"87'61","reported_seq":411,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.691590+0000","last_change":"2024-04-18T23:12:05.792408+0000","last_active":"2024-04-18T23:12:05.691590+0000","last_peered":"2024-04-18T23:12:05.691590+0000","last_clean":"2024-04-18T23:12:05.691590+0000","last_became_active":"2024-04-18T23:11:37.876250+0000","last_became_peered":"2024-04-18T23:11:37.876250+0000","last_unstale":"2024-04-18T23:12:05.691590+0000","last_undegraded":"2024-04-18T23:12:05.691590+0000","last_fullsized":"2024-04-18T23:12:05.691590+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'61","last_scrub_stamp":"2024-04-18T23:12:02.908053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:02.908053+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:10.164274+0000","scrub_duration":515,"objects_trimmed":0,"snaptrim_duration":0.101181459,"stat_sum":{"num_bytes":1949696,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":8055,"num_write":54,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"1.0","version":"33'157","reported_seq":366,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.660423+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:12:05.660423+0000","last_peered":"2024-04-18T23:12:05.660423+0000","last_clean":"2024-04-18T23:12:05.660423+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:12:05.660423+0000","last_undegraded":"2024-04-18T23:12:05.660423+0000","last_fullsized":"2024-04-18T23:12:05.660423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.690884+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:12:05.690884+0000","last_peered":"2024-04-18T23:12:05.690884+0000","last_clean":"2024-04-18T23:12:05.690884+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:12:05.690884+0000","last_undegraded":"2024-04-18T23:12:05.690884+0000","last_fullsized":"2024-04-18T23:12:05.690884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"99'93","reported_seq":429,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.865456+0000","last_change":"2024-04-18T23:12:05.867245+0000","last_active":"2024-04-18T23:12:05.865456+0000","last_peered":"2024-04-18T23:12:05.865456+0000","last_clean":"2024-04-18T23:12:05.865456+0000","last_became_active":"2024-04-18T23:11:33.818951+0000","last_became_peered":"2024-04-18T23:11:33.818951+0000","last_unstale":"2024-04-18T23:12:05.865456+0000","last_undegraded":"2024-04-18T23:12:05.865456+0000","last_fullsized":"2024-04-18T23:12:05.865456+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20639465200000001,"stat_sum":{"num_bytes":12517376,"num_objects":10,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":28,"num_read_kb":6012,"num_write":40,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":164,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.577324+0000","last_change":"2024-04-18T23:12:01.049893+0000","last_active":"2024-04-18T23:12:05.577324+0000","last_peered":"2024-04-18T23:12:05.577324+0000","last_clean":"2024-04-18T23:12:05.577324+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:12:05.577324+0000","last_undegraded":"2024-04-18T23:12:05.577324+0000","last_fullsized":"2024-04-18T23:12:05.577324+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:01.049599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.049599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:09.776200+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"88'10","reported_seq":322,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.741453+0000","last_change":"2024-04-18T23:12:05.741688+0000","last_active":"2024-04-18T23:12:05.741453+0000","last_peered":"2024-04-18T23:12:05.741453+0000","last_clean":"2024-04-18T23:12:05.741453+0000","last_became_active":"2024-04-18T23:10:54.799189+0000","last_became_peered":"2024-04-18T23:10:54.799189+0000","last_unstale":"2024-04-18T23:12:05.741453+0000","last_undegraded":"2024-04-18T23:12:05.741453+0000","last_fullsized":"2024-04-18T23:12:05.741453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.356352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035962899999999998,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":156,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.660548+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:12:05.660548+0000","last_peered":"2024-04-18T23:12:05.660548+0000","last_clean":"2024-04-18T23:12:05.660548+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:12:05.660548+0000","last_undegraded":"2024-04-18T23:12:05.660548+0000","last_fullsized":"2024-04-18T23:12:05.660548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:14.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"87'44","reported_seq":389,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.685253+0000","last_change":"2024-04-18T23:12:05.685253+0000","last_active":"2024-04-18T23:12:05.685253+0000","last_peered":"2024-04-18T23:12:05.685253+0000","last_clean":"2024-04-18T23:12:05.685253+0000","last_became_active":"2024-04-18T23:11:38.818022+0000","last_became_peered":"2024-04-18T23:11:38.818022+0000","last_unstale":"2024-04-18T23:12:05.685253+0000","last_undegraded":"2024-04-18T23:12:05.685253+0000","last_fullsized":"2024-04-18T23:12:05.685253+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10090613700000001,"stat_sum":{"num_bytes":6438912,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":2792,"num_write":20,"num_write_kb":6288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":7716864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.577536+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:12:05.577536+0000","last_peered":"2024-04-18T23:12:05.577536+0000","last_clean":"2024-04-18T23:12:05.577536+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:12:05.577536+0000","last_undegraded":"2024-04-18T23:12:05.577536+0000","last_fullsized":"2024-04-18T23:12:05.577536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.994274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":324,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.661804+0000","last_change":"2024-04-18T23:12:05.662471+0000","last_active":"2024-04-18T23:12:05.661804+0000","last_peered":"2024-04-18T23:12:05.661804+0000","last_clean":"2024-04-18T23:12:05.661804+0000","last_became_active":"2024-04-18T23:11:33.713370+0000","last_became_peered":"2024-04-18T23:11:33.713370+0000","last_unstale":"2024-04-18T23:12:05.661804+0000","last_undegraded":"2024-04-18T23:12:05.661804+0000","last_fullsized":"2024-04-18T23:12:05.661804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:10.383500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001186725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.5","version":"100'129","reported_seq":462,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.571515+0000","last_change":"2024-04-18T23:12:05.669892+0000","last_active":"2024-04-18T23:12:06.571515+0000","last_peered":"2024-04-18T23:12:06.571515+0000","last_clean":"2024-04-18T23:12:06.571515+0000","last_became_active":"2024-04-18T23:11:39.923911+0000","last_became_peered":"2024-04-18T23:11:39.923911+0000","last_unstale":"2024-04-18T23:12:06.571515+0000","last_undegraded":"2024-04-18T23:12:06.571515+0000","last_fullsized":"2024-04-18T23:12:06.571515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001107884,"stat_sum":{"num_bytes":8749056,"num_objects":10,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":12302,"num_write":64,"num_write_kb":14000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5685248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.741370+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:12:05.741370+0000","last_peered":"2024-04-18T23:12:05.741370+0000","last_clean":"2024-04-18T23:12:05.741370+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:12:05.741370+0000","last_undegraded":"2024-04-18T23:12:05.741370+0000","last_fullsized":"2024-04-18T23:12:05.741370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"98'117","reported_seq":422,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.594097+0000","last_change":"2024-04-18T23:12:05.654635+0000","last_active":"2024-04-18T23:12:06.594097+0000","last_peered":"2024-04-18T23:12:06.594097+0000","last_clean":"2024-04-18T23:12:06.594097+0000","last_became_active":"2024-04-18T23:10:54.793319+0000","last_became_peered":"2024-04-18T23:10:54.793319+0000","last_unstale":"2024-04-18T23:12:06.594097+0000","last_undegraded":"2024-04-18T23:12:06.594097+0000","last_fullsized":"2024-04-18T23:12:06.594097+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:20.152899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069787099999999997,"stat_sum":{"num_bytes":13844480,"num_objects":10,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":18,"num_read_kb":9,"num_write":40,"num_write_kb":14224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":156,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.570938+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:12:05.570938+0000","last_peered":"2024-04-18T23:12:05.570938+0000","last_clean":"2024-04-18T23:12:05.570938+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:12:05.570938+0000","last_undegraded":"2024-04-18T23:12:05.570938+0000","last_fullsized":"2024-04-18T23:12:05.570938+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.503236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"99'242","reported_seq":543,"reported_epoch":99,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:12:06.661158+0000","last_change":"2024-04-18T23:12:05.943268+0000","last_active":"2024-04-18T23:12:06.661158+0000","last_peered":"2024-04-18T23:12:06.661158+0000","last_clean":"2024-04-18T23:12:06.661158+0000","last_became_active":"2024-04-18T23:10:54.693609+0000","last_became_peered":"2024-04-18T23:10:54.693609+0000","last_unstale":"2024-04-18T23:12:06.661158+0000","last_undegraded":"2024-04-18T23:12:06.661158+0000","last_fullsized":"2024-04-18T23:12:06.661158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:13.582185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388440000000001,"stat_sum":{"num_bytes":19152896,"num_objects":20,"num_object_clones":12,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":1,"num_read":51,"num_read_kb":11706,"num_write":183,"num_write_kb":21216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.594097+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:12:06.594097+0000","last_peered":"2024-04-18T23:12:06.594097+0000","last_clean":"2024-04-18T23:12:06.594097+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:12:06.594097+0000","last_undegraded":"2024-04-18T23:12:06.594097+0000","last_fullsized":"2024-04-18T23:12:06.594097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"87'42","reported_seq":391,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.752978+0000","last_change":"2024-04-18T23:12:05.853594+0000","last_active":"2024-04-18T23:12:05.752978+0000","last_peered":"2024-04-18T23:12:05.752978+0000","last_clean":"2024-04-18T23:12:05.752978+0000","last_became_active":"2024-04-18T23:11:39.717443+0000","last_became_peered":"2024-04-18T23:11:39.717443+0000","last_unstale":"2024-04-18T23:12:05.752978+0000","last_undegraded":"2024-04-18T23:12:05.752978+0000","last_fullsized":"2024-04-18T23:12:05.752978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:13.038125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.100779232,"stat_sum":{"num_bytes":3358720,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":7544,"num_write":18,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.571293+0000","last_change":"2024-04-18T23:11:33.519671+0000","last_active":"2024-04-18T23:12:06.571293+0000","last_peered":"2024-04-18T23:12:06.571293+0000","last_clean":"2024-04-18T23:12:06.571293+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:12:06.571293+0000","last_undegraded":"2024-04-18T23:12:06.571293+0000","last_fullsized":"2024-04-18T23:12:06.571293+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":135921664,"num_objects":129,"num_object_clones":79,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":5,"num_read":448,"num_read_kb":146633,"num_write":841,"num_write_kb":152928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237445120,"data_stored":362332160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0},"log_size":1431,"ondisk_log_size":1431,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808840,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39508,"kb_used_data":10940,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94332332,"statfs":{"total":96636764160,"available":96596307968,"internally_reserved":0,"allocated":11202560,"data_stored":16809255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280533,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44024,"kb_used_data":12840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327816,"statfs":{"total":96636764160,"available":96591683584,"internally_reserved":0,"allocated":13148160,"data_stored":17163444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280533,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42440,"kb_used_data":11260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329400,"statfs":{"total":96636764160,"available":96593305600,"internally_reserved":0,"allocated":11530240,"data_stored":15320788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313236,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58156,"kb_used_data":26976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94313684,"statfs":{"total":96636764160,"available":96577212416,"internally_reserved":0,"allocated":27623424,"data_stored":40629428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313236,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49452,"kb_used_data":18272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322388,"statfs":{"total":96636764160,"available":96586125312,"internally_reserved":0,"allocated":18710528,"data_stored":29672628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313237,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":9996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":10235904,"data_stored":13665460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313237,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54124,"kb_used_data":22944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94317716,"statfs":{"total":96636764160,"available":96581341184,"internally_reserved":0,"allocated":23494656,"data_stored":34436276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313237,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37620,"kb_used_data":10596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334220,"statfs":{"total":96636764160,"available":96598241280,"internally_reserved":0,"allocated":10850304,"data_stored":14701748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52212,"kb_used_data":21032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94319628,"statfs":{"total":96636764160,"available":96583299072,"internally_reserved":0,"allocated":21536768,"data_stored":31421620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313235,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45120,"kb_used_data":13940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326720,"statfs":{"total":96636764160,"available":96590561280,"internally_reserved":0,"allocated":14274560,"data_stored":20796596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313236,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42456,"kb_used_data":11276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329384,"statfs":{"total":96636764160,"available":96593289216,"internally_reserved":0,"allocated":11546624,"data_stored":18537531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280533,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43780,"kb_used_data":12604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328060,"statfs":{"total":96636764160,"available":96591933440,"internally_reserved":0,"allocated":12906496,"data_stored":17960539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280533,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54620,"kb_used_data":23440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94317220,"statfs":{"total":96636764160,"available":96580833280,"internally_reserved":0,"allocated":24002560,"data_stored":36902068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280534,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47460,"kb_used_data":16272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324380,"statfs":{"total":96636764160,"available":96588165120,"internally_reserved":0,"allocated":16662528,"data_stored":24136999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280533,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46360,"kb_used_data":15168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325480,"statfs":{"total":96636764160,"available":96589291520,"internally_reserved":0,"allocated":15532032,"data_stored":23416103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280533,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41460,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330380,"statfs":{"total":96636764160,"available":96594309120,"internally_reserved":0,"allocated":10518528,"data_stored":14089511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26161152,"data_stored":39284736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":31440896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10928128,"data_stored":16146432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8773632,"data_stored":12369920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22032384,"data_stored":33091584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9388032,"data_stored":13844480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":22540288,"data_stored":36044800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15183872,"data_stored":23269376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20787200,"data_stored":32374784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16502784,"data_stored":25726976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":10350592,"data_stored":14532608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12812288,"data_stored":19218432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":11591680,"data_stored":19181568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":9535488,"data_stored":13496320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":11685888,"data_stored":16306176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":9785344,"data_stored":16003072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0}]}} 2024-04-18T23:12:09.637 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:09.636+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:09.704 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:09.704 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:09.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:12:09.743 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:12:09.744 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:12:09.744 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:12:09.742+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:12:09.817 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:12:09.818 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:12:09.824 INFO:tasks.rados.rados.0.smithi142.stdout:395: expect (ObjNum 0 snap 0 seq_num 0) 2024-04-18T23:12:09.831 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:09.832 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:09.838 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:12:09.834+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:12:09.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:12:09.869 INFO:tasks.rados.rados.0.smithi142.stdout:396: finishing write tid 1 to smithi14230322-21 2024-04-18T23:12:09.869 INFO:tasks.rados.rados.0.smithi142.stdout:396: finishing write tid 2 to smithi14230322-21 2024-04-18T23:12:09.870 INFO:tasks.rados.rados.0.smithi142.stdout:396: finishing write tid 3 to smithi14230322-21 2024-04-18T23:12:09.928 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:12:09.928 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:12:09.939 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:12:09.934+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:12:09.955 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:12:09.956 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:12:10.006 INFO:tasks.rados.rados.0.smithi142.stdout:394: finishing rollback tid 0 to smithi14230322-32 2024-04-18T23:12:10.006 INFO:tasks.rados.rados.0.smithi142.stdout:394: finishing rollback tid 1 to smithi14230322-32 2024-04-18T23:12:10.007 INFO:tasks.rados.rados.0.smithi142.stdout:394: finishing rollback tid 2 to smithi14230322-32 2024-04-18T23:12:10.007 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 32 v 252 (ObjNum 119 snap 26 seq_num 119) dirty exists 2024-04-18T23:12:10.009 INFO:tasks.rados.rados.0.smithi142.stdout:397: finishing write tid 1 to smithi14230322-4 2024-04-18T23:12:10.009 INFO:tasks.rados.rados.0.smithi142.stdout:397: finishing write tid 2 to smithi14230322-4 2024-04-18T23:12:10.010 INFO:tasks.rados.rados.0.smithi142.stdout:397: finishing write tid 3 to smithi14230322-4 2024-04-18T23:12:10.011 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:10.011 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:10.011 INFO:tasks.rados.rados.0.smithi142.stdout:397: finishing write tid 4 to smithi14230322-4 2024-04-18T23:12:10.012 INFO:tasks.rados.rados.0.smithi142.stdout:397: finishing write tid 5 to smithi14230322-4 2024-04-18T23:12:10.012 INFO:tasks.rados.rados.0.smithi142.stdout:394: done (4 left) 2024-04-18T23:12:10.012 INFO:tasks.rados.rados.0.smithi142.stdout:395: done (3 left) 2024-04-18T23:12:10.012 INFO:tasks.rados.rados.0.smithi142.stdout:398: done (2 left) 2024-04-18T23:12:10.012 INFO:tasks.rados.rados.0.smithi142.stdout:append oid 5 current snap is 38 2024-04-18T23:12:10.012 INFO:tasks.rados.rados.0.smithi142.stdout:399: seq_num 142 ranges {3506176=442368,3948544=720896,4669440=540672,5210112=688128} 2024-04-18T23:12:10.026 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:12:10.033 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:12:10.034 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:12:10.035 INFO:tasks.rados.rados.0.smithi142.stdout:399: writing smithi14230322-5 from 3506176 to 3948544 tid 1 2024-04-18T23:12:10.040 INFO:tasks.rados.rados.0.smithi142.stdout:399: writing smithi14230322-5 from 3948544 to 4669440 tid 2 2024-04-18T23:12:10.043 INFO:tasks.rados.rados.0.smithi142.stdout:399: writing smithi14230322-5 from 4669440 to 5210112 tid 3 2024-04-18T23:12:10.047 INFO:tasks.rados.rados.0.smithi142.stdout:399: writing smithi14230322-5 from 5210112 to 5898240 tid 4 2024-04-18T23:12:10.047 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 3 2024-04-18T23:12:10.063 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:12:10.064 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:12:10.064 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:12:10.062+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:12:10.091 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:12:10.092 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:12:10.120 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:12:10.120 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:12:10.140 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:12:10.138+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:12:10.143 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:12:10.144 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:12:10.154 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:10.154 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:12:10.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:12:10.154 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:12:10.154 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.154 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:194 3.4s0 3:2a56ff57:::smithi14230322-24:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:10.154 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:12:10.154 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 69.592683004999998, 2024-04-18T23:12:10.154 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41191497500000002, 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 194 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.155 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558779+0000", 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558781+0000", 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8059999999999999e-06 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558789+0000", 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1340000000000004e-06 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.156 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558806+0000", 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6300999999999999e-05 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558814+0000", 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0099999999999995e-06 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558882+0000", 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8281999999999994e-05 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.558889+0000", 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2359999999999998e-06 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661030+0000", 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102141155 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.661144+0000", 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113853 2024-04-18T23:12:10.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.766467+0000", 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.105323126 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.768352+0000", 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0018843359999999999 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.970633+0000", 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.202281391 2024-04-18T23:12:10.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:00.970694+0000", 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1345000000000003e-05 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:469 3.5s1 3:a09a6894:::smithi14230322-25:head [copy-from ver 15] snapc 11=[11,10,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.635166+0000", 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.516296126999997, 2024-04-18T23:12:10.160 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.82619366100000002, 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 469 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635166+0000", 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635166+0000", 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635169+0000", 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0400000000000001e-06 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635184+0000", 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4926000000000001e-05 2024-04-18T23:12:10.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635212+0000", 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7597e-05 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635225+0000", 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3643e-05 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635366+0000", 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014097400000000001 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "op must wait for map", 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635381+0000", 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4331000000000001e-05 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.638693+0000", 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0033120110000000001 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.638809+0000", 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000115821 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.957052+0000", 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.31824379899999999 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.961730+0000", 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0046779459999999997 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.460885+0000", 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.49915458400000001 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.165 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.461360+0000", 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00047498899999999999 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:471 3.5s1 3:a832c25e:::smithi14230322-46:head [copy-from ver 21] snapc 11=[11,10,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.635334+0000", 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.516128117000001, 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.52312541499999998, 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.166 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 471 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635334+0000", 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635334+0000", 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635335+0000", 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5199999999999996e-07 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635340+0000", 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4929999999999998e-06 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635351+0000", 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0845e-05 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.168 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635358+0000", 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1559999999999998e-06 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635569+0000", 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00021029600000000001 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.635581+0000", 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2456000000000001e-05 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.639267+0000", 2024-04-18T23:12:10.169 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0036858080000000001 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.639421+0000", 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015406999999999999 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.844361+0000", 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20493995800000001 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.846386+0000", 2024-04-18T23:12:10.170 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020250310000000001 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.158122+0000", 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.31173636799999999 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.158460+0000", 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000337282 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs1 61/31 ECSubWrite(tid=42, reqid=osd.11.0:41, at_version=0'0, trim_to=0'0, roll_forward_to=61'14))", 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.958302+0000", 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.193160659, 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38844035799999999, 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958302+0000", 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958302+0000", 2024-04-18T23:12:10.172 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958306+0000", 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3170000000000003e-06 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958413+0000", 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000107431 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958531+0000", 2024-04-18T23:12:10.173 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000117197 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.958575+0000", 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4651999999999999e-05 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.143952+0000", 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.18537637400000001 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.174 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.143979+0000", 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7772000000000001e-05 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.346690+0000", 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.202710693 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.346742+0000", 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.1922000000000002e-05 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:548 3.5s1 3:a6fe874e:::smithi14230322-33:head [append 0~0] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.531316+0000", 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.620146439000003, 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.42558847100000002, 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 548 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531316+0000", 2024-04-18T23:12:10.176 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531316+0000", 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531318+0000", 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9039999999999999e-06 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531328+0000", 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7599999999999997e-06 2024-04-18T23:12:10.177 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531371+0000", 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3377999999999997e-05 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531383+0000", 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.223e-05 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547544+0000", 2024-04-18T23:12:10.178 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.016161225000000001 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547703+0000", 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015882900000000001 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.548679+0000", 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00097595699999999995 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.756002+0000", 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20732305100000001 2024-04-18T23:12:10.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956635+0000", 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20063264 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956904+0000", 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000269497 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:549 3.5s1 3:a6fe874e:::smithi14230322-33:head [rollback 13] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.531403+0000", 2024-04-18T23:12:10.180 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.620059052000002, 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.52622381100000004, 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 549 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.181 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531403+0000", 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531403+0000", 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531404+0000", 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8599999999999998e-07 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.182 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531409+0000", 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0780000000000003e-06 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531417+0000", 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5499999999999997e-06 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531424+0000", 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1570000000000003e-06 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.183 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.549557+0000", 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.018133732999999999 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.549692+0000", 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013470600000000001 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.550566+0000", 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00087358599999999998 2024-04-18T23:12:10.184 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.856154+0000", 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30558865499999999 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057374+0000", 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20122003399999999 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057627+0000", 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025272599999999999 2024-04-18T23:12:10.185 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:550 3.5s1 3:a6fe874e:::smithi14230322-33:head [append 0~0] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.531438+0000", 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.620023985000003, 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.92696160500000002, 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.186 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 550 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531438+0000", 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531438+0000", 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.187 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531439+0000", 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7100000000000002e-07 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531441+0000", 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6419999999999999e-06 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531448+0000", 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.844e-06 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.531453+0000", 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0459999999999996e-06 2024-04-18T23:12:10.188 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.058137+0000", 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.52668325500000002 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.058247+0000", 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000110487 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.058710+0000", 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00046313300000000001 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.189 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.062805+0000", 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.004094749 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.457788+0000", 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39498330100000001 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.458400+0000", 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00061167700000000003 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.190 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.5s1 69/68 ECSubRead(tid=76, to_read={3:a832c25e:::smithi14230322-46:11=0,262144,0}, subchunks={3:a832c25e:::smithi14230322-46:11=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.604064657999999, 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.51415498900000001, 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.191 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.547398+0000", 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.192 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.550808+0000", 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.003410153 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.550820+0000", 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1797000000000001e-05 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.059003+0000", 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50818370400000001 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.193 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.061553+0000", 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025493349999999998 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=78, last_complete=69'41, committed=1, applied=1))", 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.555925+0000", 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.595537229000001, 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40055818999999998, 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.194 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555925+0000", 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555925+0000", 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555926+0000", 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.015e-06 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.195 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555930+0000", 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6990000000000001e-06 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555945+0000", 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5007999999999999e-05 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.555957+0000", 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.19e-05 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.196 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956440+0000", 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.400482909 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.956483+0000", 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3659000000000003e-05 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=78, last_complete=69'41, committed=1, applied=1))", 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.633355+0000", 2024-04-18T23:12:10.197 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.518107120000003, 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.424701157, 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633355+0000", 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633355+0000", 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.198 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633356+0000", 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.0999999999999999e-07 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633360+0000", 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7919999999999999e-06 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633370+0000", 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0088000000000001e-05 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.633378+0000", 2024-04-18T23:12:10.199 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9899999999999997e-06 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057298+0000", 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.42392059799999998 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.058056+0000", 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00075817899999999999 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 69/68 ECSubWriteReply(tid=78, last_complete=69'41, committed=1, applied=1))", 2024-04-18T23:12:10.200 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.651505+0000", 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.499957162000001, 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40577508200000001, 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651505+0000", 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651505+0000", 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.201 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651506+0000", 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7199999999999999e-07 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651509+0000", 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.641e-06 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651518+0000", 2024-04-18T23:12:10.202 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2940000000000006e-06 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.651523+0000", 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9019999999999998e-06 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057243+0000", 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40572015299999997 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057280+0000", 2024-04-18T23:12:10.203 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.7620000000000002e-05 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWriteReply(3.5s1 70/68 ECSubWriteReply(tid=78, last_complete=69'41, committed=1, applied=1))", 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:36.655398+0000", 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 33.496064390999997, 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40178138000000002, 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.204 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655398+0000", 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655398+0000", 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655399+0000", 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1009999999999999e-06 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.205 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655406+0000", 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5300000000000002e-06 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655420+0000", 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4824000000000001e-05 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:36.655436+0000", 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5600999999999999e-05 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.206 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057155+0000", 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40171905899999999 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:37.057179+0000", 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4264999999999999e-05 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:921 3.4s0 3:2a56ff57:::smithi14230322-24:head [append 0~720896] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:10.207 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:04.622672+0000", 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.5287906969999998, 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50777757499999998, 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 921 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.622672+0000", 2024-04-18T23:12:10.208 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.622672+0000", 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.622673+0000", 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.39e-06 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.623468+0000", 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00079528699999999999 2024-04-18T23:12:10.209 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.623494+0000", 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5338999999999999e-05 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.723577+0000", 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008355400000001 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.723984+0000", 2024-04-18T23:12:10.210 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00040720600000000002 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724135+0000", 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015057300000000001 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.725353+0000", 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012181780000000001 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.731971+0000", 2024-04-18T23:12:10.211 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0066177179999999999 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.130299+0000", 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39832843699999998 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.130449+0000", 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014989300000000001 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.212 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:922 3.4s0 3:2a56ff57:::smithi14230322-24:head [append 0~475136] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:04.723615+0000", 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.4278470389999995, 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41198226500000001, 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 922 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.213 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.723615+0000", 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.723615+0000", 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.723617+0000", 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4720000000000001e-06 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.214 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724043+0000", 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00042651899999999999 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724057+0000", 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3652000000000001e-05 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724065+0000", 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7419999999999997e-06 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.726694+0000", 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0026293010000000001 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.726823+0000", 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000128835 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.727718+0000", 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00089530999999999996 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.733063+0000", 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.005345224 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.135466+0000", 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40240216699999998 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.135598+0000", 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00013204300000000001 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.217 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:923 3.4s0 3:2a56ff57:::smithi14230322-24:head [append 0~360448] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:04.724077+0000", 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.4273852680000001, 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.511938326, 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 923 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.218 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724077+0000", 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724077+0000", 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724078+0000", 2024-04-18T23:12:10.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3099999999999997e-07 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724393+0000", 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000315433 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724404+0000", 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0827e-05 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724411+0000", 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4089999999999997e-06 2024-04-18T23:12:10.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.730236+0000", 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0058247029999999997 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.730344+0000", 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010772000000000001 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.731216+0000", 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00087206299999999996 2024-04-18T23:12:10.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.733743+0000", 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025273460000000002 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.235765+0000", 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.50202188700000006 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.236015+0000", 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00025020700000000001 2024-04-18T23:12:10.222 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:924 3.4s0 3:2a56ff57:::smithi14230322-24:head [setxattr _header (57) in=7b] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:04.724423+0000", 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.4270391560000002, 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.61245996199999997, 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.223 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 924 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724423+0000", 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724423+0000", 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724424+0000", 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8800000000000002e-07 2024-04-18T23:12:10.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724428+0000", 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6830000000000001e-06 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.724437+0000", 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4040000000000001e-06 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.824574+0000", 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10013654299999999 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.924885+0000", 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10031150899999999 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.925076+0000", 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000190595 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.925914+0000", 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00083876600000000001 2024-04-18T23:12:10.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.927937+0000", 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0020223950000000002 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.336598+0000", 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40866082599999998 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.336883+0000", 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028555300000000002 2024-04-18T23:12:10.227 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:925 3.4s0 3:2a56ff57:::smithi14230322-24:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:04.824612+0000", 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 5.326850232, 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71561323399999999, 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 925 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.228 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.824612+0000", 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.824612+0000", 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.824614+0000", 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.72e-06 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.824622+0000", 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5669999999999995e-06 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.824643+0000", 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0312000000000001e-05 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.824659+0000", 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6237000000000001e-05 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.926630+0000", 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10197079000000001 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:04.926896+0000", 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00026636299999999998 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.337963+0000", 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.41106699800000002 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.338120+0000", 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00015726600000000001 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:05.540225+0000", 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20210498099999999 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s4 99/31 ECSubWrite(tid=462, reqid=osd.4.0:460, at_version=0'0, trim_to=0'0, roll_forward_to=99'242))", 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:06.661687+0000", 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.4897752039999999, 2024-04-18T23:12:10.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40194740699999998, 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.661687+0000", 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.661687+0000", 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.661689+0000", 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.063e-06 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.661720+0000", 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0926000000000002e-05 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.661759+0000", 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9189e-05 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.661785+0000", 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.605e-05 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.862093+0000", 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20030805500000001 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.862110+0000", 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7003999999999998e-05 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.063585+0000", 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201474976 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.063635+0000", 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9144000000000002e-05 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s4 99/31 ECSubWrite(tid=463, reqid=osd.4.0:463, at_version=99'244, trim_to=0'0, roll_forward_to=99'242))", 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:06.663335+0000", 2024-04-18T23:12:10.236 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.4881270230000001, 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.400334894, 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.663335+0000", 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.663335+0000", 2024-04-18T23:12:10.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.663336+0000", 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.06e-06 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.663399+0000", 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3045000000000004e-05 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.663432+0000", 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2302999999999999e-05 2024-04-18T23:12:10.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.663453+0000", 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1739e-05 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.962608+0000", 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.29915497099999999 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.962617+0000", 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.337e-06 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.063642+0000", 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101025367 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.063670+0000", 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8072000000000002e-05 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.7s4 99/31 ECSubWrite(tid=464, reqid=osd.4.0:464, at_version=99'246, trim_to=0'0, roll_forward_to=99'242))", 2024-04-18T23:12:10.240 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:06.664929+0000", 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.4865331930000001, 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39922748600000002, 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.664929+0000", 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.664929+0000", 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.664931+0000", 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4529999999999999e-06 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.664984+0000", 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3474999999999999e-05 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.665016+0000", 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2013e-05 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.665037+0000", 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0625999999999999e-05 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.063128+0000", 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39809090400000002 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.063156+0000", 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8838000000000001e-05 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.064126+0000", 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00096946800000000002 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.064157+0000", 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0709e-05 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.244 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:10.245 INFO:tasks.rados.rados.0.smithi142.stdout:397: finishing write tid 6 to smithi14230322-4 2024-04-18T23:12:10.246 INFO:tasks.rados.rados.0.smithi142.stdout:397: oid 4 updating version 0 to 110 2024-04-18T23:12:10.246 INFO:tasks.rados.rados.0.smithi142.stdout:397: oid 4 updating version 110 to 112 2024-04-18T23:12:10.246 INFO:tasks.rados.rados.0.smithi142.stdout:397: oid 4 version 112 is already newer than 111 2024-04-18T23:12:10.246 INFO:tasks.rados.rados.0.smithi142.stdout:397: oid 4 updating version 112 to 113 2024-04-18T23:12:10.246 INFO:tasks.rados.rados.0.smithi142.stdout:397: oid 4 updating version 113 to 114 2024-04-18T23:12:10.246 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 4 v 114 (ObjNum 140 snap 38 seq_num 140) dirty exists 2024-04-18T23:12:10.246 INFO:tasks.rados.rados.0.smithi142.stdout:397: left oid 4 (ObjNum 140 snap 38 seq_num 140) 2024-04-18T23:12:10.246 INFO:tasks.rados.rados.0.smithi142.stdout:397: done (2 left) 2024-04-18T23:12:10.246 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 2 2024-04-18T23:12:10.247 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:12:10.247 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:12:10.247 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:12:10.248 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:12:10.248 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:12:10.248 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:10.242+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:10.276 INFO:tasks.rados.rados.0.smithi142.stdout:396: finishing write tid 4 to smithi14230322-21 2024-04-18T23:12:10.277 INFO:tasks.rados.rados.0.smithi142.stdout:396: finishing write tid 5 to smithi14230322-21 2024-04-18T23:12:10.301 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:12:10.301 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:12:10.321 INFO:tasks.rados.rados.0.smithi142.stdout:399: finishing write tid 1 to smithi14230322-5 2024-04-18T23:12:10.325 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:12:10.326 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:12:10.346 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:10.347+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:10.351 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:12:10.351 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:12:10.379 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:12:10.379 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:12:10.405 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:12:10.405 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:12:10.411 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:10.412 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:12:10.412 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:12:10.412 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:10.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:12:10.447 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:10.447+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:10.523 INFO:tasks.rados.rados.0.smithi142.stdout:399: finishing write tid 2 to smithi14230322-5 2024-04-18T23:12:10.523 INFO:tasks.rados.rados.0.smithi142.stdout:399: finishing write tid 3 to smithi14230322-5 2024-04-18T23:12:10.524 INFO:tasks.rados.rados.0.smithi142.stdout:399: finishing write tid 4 to smithi14230322-5 2024-04-18T23:12:10.548 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:10.548+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:10.584 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:10.585 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:12:10.585 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:12:10.585 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:12:10.585 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:10.598 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:12:10.623 INFO:tasks.rados.rados.0.smithi142.stdout:399: finishing write tid 5 to smithi14230322-5 2024-04-18T23:12:10.649 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:12:10.646+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:12:10.680 INFO:tasks.rados.rados.0.smithi142.stdout:396: finishing write tid 6 to smithi14230322-21 2024-04-18T23:12:10.680 INFO:tasks.rados.rados.0.smithi142.stdout:396: oid 21 updating version 0 to 185 2024-04-18T23:12:10.681 INFO:tasks.rados.rados.0.smithi142.stdout:396: oid 21 version 185 is already newer than 184 2024-04-18T23:12:10.681 INFO:tasks.rados.rados.0.smithi142.stdout:396: oid 21 updating version 185 to 188 2024-04-18T23:12:10.681 INFO:tasks.rados.rados.0.smithi142.stdout:396: oid 21 version 188 is already newer than 186 2024-04-18T23:12:10.681 INFO:tasks.rados.rados.0.smithi142.stdout:396: oid 21 version 188 is already newer than 187 2024-04-18T23:12:10.681 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 21 v 188 (ObjNum 139 snap 38 seq_num 139) dirty exists 2024-04-18T23:12:10.681 INFO:tasks.rados.rados.0.smithi142.stdout:396: left oid 21 (ObjNum 139 snap 38 seq_num 139) 2024-04-18T23:12:10.681 INFO:tasks.rados.rados.0.smithi142.stdout:396: done (1 left) 2024-04-18T23:12:10.681 INFO:tasks.rados.rados.0.smithi142.stdout: waiting on 1 2024-04-18T23:12:10.718 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 42/31 ECSubWrite(tid=85, reqid=osd.4.0:85, at_version=42'27, trim_to=0'0, roll_forward_to=40'25))", 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 64.139750367000005, 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16837443799999999, 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.719 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577732+0000", 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577733+0000", 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6300000000000001e-07 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.720 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577758+0000", 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5099000000000001e-05 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577779+0000", 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0937999999999998e-05 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.577792+0000", 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3227999999999999e-05 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.721 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744736+0000", 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.166943963 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.744769+0000", 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2629000000000002e-05 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746075+0000", 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001306338 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:06.746107+0000", 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.188e-05 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 47/31 ECSubWrite(tid=26, reqid=client.4568.0:333, at_version=47'16, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.985598605, 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201860068, 2024-04-18T23:12:10.723 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.731884+0000", 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6299999999999999e-07 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732265+0000", 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00038060399999999999 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732281+0000", 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6362e-05 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.732292+0000", 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0817e-05 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.832471+0000", 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100179254 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.832486+0000", 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4911e-05 2024-04-18T23:12:10.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933708+0000", 2024-04-18T23:12:10.727 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101221567 2024-04-18T23:12:10.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.727 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.727 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933744+0000", 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6390000000000002e-05 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 47/31 ECSubWrite(tid=27, reqid=client.4568.0:344, at_version=47'18, trim_to=0'0, roll_forward_to=42'14))", 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 57.982951270999997, 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19924303900000001, 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734531+0000", 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734532+0000", 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7299999999999995e-07 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734555+0000", 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3258999999999999e-05 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734568+0000", 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2568999999999999e-05 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.734575+0000", 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6279999999999999e-06 2024-04-18T23:12:10.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.833597+0000", 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099021993000000003 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.833605+0000", 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0620000000000001e-06 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933755+0000", 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10015038 2024-04-18T23:12:10.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:12.933774+0000", 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9474999999999999e-05 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 60/31 ECSubWrite(tid=38, reqid=client.4568.0:452, at_version=0'0, trim_to=0'0, roll_forward_to=60'10))", 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.352985+0000", 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 44.364498124999997, 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30133912600000001, 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.732 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.352985+0000", 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.352985+0000", 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.733 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.352986+0000", 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0109999999999999e-06 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.353002+0000", 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6235999999999998e-05 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.353020+0000", 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8451999999999999e-05 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.734 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.453102+0000", 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100081295 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.553350+0000", 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100248296 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.553384+0000", 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3729000000000002e-05 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.735 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654230+0000", 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100846175 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.654324+0000", 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3931999999999994e-05 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 61/31 ECSubWrite(tid=43, reqid=osd.2.0:43, at_version=61'31, trim_to=0'0, roll_forward_to=60'27))", 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:26.847226+0000", 2024-04-18T23:12:10.736 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 43.870256226999999, 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201633386, 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847226+0000", 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847226+0000", 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847227+0000", 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.0999999999999997e-07 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847269+0000", 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.2058999999999998e-05 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.847292+0000", 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2299000000000001e-05 2024-04-18T23:12:10.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:26.947376+0000", 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008486599999999 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.047557+0000", 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10018052099999999 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.047574+0000", 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6690000000000001e-05 2024-04-18T23:12:10.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.048752+0000", 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011779469999999999 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:27.048860+0000", 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000108194 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 72/31 ECSubWrite(tid=65, reqid=osd.2.0:65, at_version=72'50, trim_to=0'0, roll_forward_to=71'48))", 2024-04-18T23:12:10.740 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.617025+0000", 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.100457218000003, 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20153253099999999, 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617025+0000", 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617025+0000", 2024-04-18T23:12:10.741 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617026+0000", 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.8999999999999996e-07 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617083+0000", 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7328999999999999e-05 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.742 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617107+0000", 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4099000000000001e-05 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.617124+0000", 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6310000000000001e-05 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.817365+0000", 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20024151300000001 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.743 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.817375+0000", 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.8660000000000001e-06 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.818514+0000", 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011390809999999999 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.818558+0000", 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.3742999999999998e-05 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.744 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.es6 72/31 ECSubWrite(tid=66, reqid=osd.2.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.618834+0000", 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.098648503, 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20101856300000001, 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618834+0000", 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.745 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618834+0000", 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618835+0000", 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.746 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618863+0000", 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.7520000000000001e-05 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.618882+0000", 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9066e-05 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.718989+0000", 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100107504 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.747 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.817858+0000", 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.098869033999999995 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.817869+0000", 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0997e-05 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819809+0000", 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019393909999999999 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.748 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819853+0000", 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.4155e-05 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 72/31 ECSubWrite(tid=60, reqid=client.4568.0:561, at_version=72'28, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.630952+0000", 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.086530979000003, 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.174762375, 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.749 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.630952+0000", 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.630952+0000", 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.630952+0000", 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.6700000000000003e-07 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.750 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.631436+0000", 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00048351399999999999 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.631467+0000", 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1149999999999998e-05 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.631483+0000", 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6348999999999999e-05 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.751 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.704225+0000", 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.072741410000000006 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.704236+0000", 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0828e-05 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805658+0000", 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101421936 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.752 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805714+0000", 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6421000000000001e-05 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 72/31 ECSubWrite(tid=61, reqid=osd.11.0:61, at_version=72'30, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.634135+0000", 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.083347789999998, 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.17163550899999999, 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.753 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634135+0000", 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634135+0000", 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634136+0000", 2024-04-18T23:12:10.754 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3829999999999999e-06 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634172+0000", 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.5886000000000002e-05 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634197+0000", 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5018999999999999e-05 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.634214+0000", 2024-04-18T23:12:10.755 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6439999999999998e-05 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.704741+0000", 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.070527272000000002 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.704748+0000", 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5719999999999997e-06 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805721+0000", 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100973006 2024-04-18T23:12:10.756 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805770+0000", 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8931000000000003e-05 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 72/31 ECSubWrite(tid=63, reqid=osd.11.0:63, at_version=72'32, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.636571+0000", 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 32.080911325000002, 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.169977613, 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.757 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636571+0000", 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636571+0000", 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.758 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636573+0000", 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5689999999999999e-06 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636610+0000", 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6946000000000002e-05 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636646+0000", 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6298999999999999e-05 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.759 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.636667+0000", 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0429999999999999e-05 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805168+0000", 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.16850090300000001 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.805182+0000", 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4132e-05 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.760 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.806507+0000", 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001325718 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.806549+0000", 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1616000000000003e-05 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.761 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 72/31 ECSubWrite(tid=215, reqid=osd.4.0:215, at_version=72'74, trim_to=0'0, roll_forward_to=71'72))", 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:38.719128+0000", 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 31.998354745, 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20248240000000001, 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719128+0000", 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.762 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719128+0000", 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719129+0000", 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.6700000000000003e-07 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719183+0000", 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.4279e-05 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.763 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.719207+0000", 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4586000000000001e-05 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.819295+0000", 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10008806200000001 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.919464+0000", 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100168431 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.764 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.919480+0000", 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5722999999999998e-05 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.921452+0000", 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001972054 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:38.921610+0000", 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000158698 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.765 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 81/31 ECSubWrite(tid=309, reqid=osd.4.0:305, at_version=0'0, trim_to=0'0, roll_forward_to=81'120))", 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:48.545814+0000", 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 22.171668641, 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20083568399999999, 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.766 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545814+0000", 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545814+0000", 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545815+0000", 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3969999999999999e-06 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545853+0000", 2024-04-18T23:12:10.767 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.8000999999999998e-05 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545874+0000", 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0068000000000002e-05 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.545891+0000", 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7176999999999998e-05 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.768 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.646066+0000", 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10017543700000001 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.646074+0000", 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9389999999999997e-06 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.746609+0000", 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100534928 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.769 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:48.746650+0000", 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0737e-05 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpRead(3.bs3 84/31 ECSubRead(tid=332, to_read={3:d447fef0:::smithi14230322-49:head=0,4096,0}, subchunks={3:d447fef0:::smithi14230322-49:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.759540+0000", 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 19.957942709000001, 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.200539947, 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.770 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759540+0000", 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759540+0000", 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.771 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759541+0000", 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9999999999999997e-07 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759548+0000", 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.5000000000000002e-06 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.759559+0000", 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0587999999999999e-05 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.859637+0000", 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100078449 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.959780+0000", 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100142606 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.960080+0000", 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00030010399999999998 2024-04-18T23:12:10.773 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 94/31 ECSubWrite(tid=404, reqid=client.4568.0:825, at_version=94'165, trim_to=0'0, roll_forward_to=92'163))", 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.551379+0000", 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.166104064000001, 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20496428999999999, 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.774 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.551379+0000", 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.551379+0000", 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.551379+0000", 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4099999999999998e-07 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.775 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.551544+0000", 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000164809 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.551570+0000", 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5761e-05 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.551588+0000", 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8266000000000001e-05 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.776 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.651856+0000", 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100267521 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.651873+0000", 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7312999999999998e-05 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.756250+0000", 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10437740099999999 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.777 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.756343+0000", 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.2577999999999994e-05 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 94/31 ECSubWrite(tid=405, reqid=osd.4.0:405, at_version=94'167, trim_to=0'0, roll_forward_to=92'163))", 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.557732+0000", 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.159751118000001, 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19868377500000001, 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.778 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557732+0000", 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557732+0000", 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557733+0000", 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7699999999999992e-07 2024-04-18T23:12:10.779 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557766+0000", 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.3056999999999999e-05 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557787+0000", 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.1531000000000001e-05 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.557801+0000", 2024-04-18T23:12:10.780 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3468e-05 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.754220+0000", 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19641951399999999 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.754237+0000", 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6529e-05 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.756360+0000", 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00212363 2024-04-18T23:12:10.781 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.756415+0000", 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5068999999999997e-05 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 94/31 ECSubWrite(tid=406, reqid=client.4568.0:826, at_version=94'168, trim_to=0'0, roll_forward_to=92'163))", 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.558809+0000", 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.158673524999999, 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19811614599999999, 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.782 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.558809+0000", 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.558809+0000", 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.783 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.558810+0000", 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.074e-06 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.558979+0000", 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016832899999999999 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.558996+0000", 2024-04-18T23:12:10.784 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7138000000000001e-05 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.559005+0000", 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.6069999999999997e-06 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.754856+0000", 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.19585081400000001 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.785 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.754888+0000", 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1921999999999997e-05 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.756853+0000", 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0019646030000000001 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.756925+0000", 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2658999999999997e-05 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.786 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 94/31 ECSubWrite(tid=407, reqid=client.4568.0:827, at_version=94'169, trim_to=0'0, roll_forward_to=92'163))", 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:00.761334+0000", 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.9561491390000008, 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20135033599999999, 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761334+0000", 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.787 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761334+0000", 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761336+0000", 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3329999999999999e-06 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761671+0000", 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00033465900000000001 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.788 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.761720+0000", 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9678000000000001e-05 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.861816+0000", 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100096059 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.861901+0000", 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.5159000000000002e-05 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.789 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.861914+0000", 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2969999999999999e-05 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.962626+0000", 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10071161300000001 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:00.962684+0000", 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7865000000000002e-05 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.790 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 94/31 ECSubWrite(tid=413, reqid=client.4568.0:832, at_version=94'174, trim_to=0'0, roll_forward_to=94'173))", 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.273909+0000", 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.4435738279999999, 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.200626674, 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.273909+0000", 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.791 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.273909+0000", 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.273911+0000", 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.2529999999999999e-06 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.273961+0000", 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9666000000000002e-05 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.792 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.274001+0000", 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9999000000000001e-05 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.274050+0000", 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.9539000000000001e-05 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.372574+0000", 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.098524082999999998 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.793 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.372598+0000", 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.3773000000000001e-05 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.474458+0000", 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101859743 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.474536+0000", 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7618000000000001e-05 2024-04-18T23:12:10.794 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs3 94/31 ECSubWrite(tid=414, reqid=client.4568.0:833, at_version=94'175, trim_to=0'0, roll_forward_to=94'173))", 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.276512+0000", 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 9.4409705069999994, 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.198068402, 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.795 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.276512+0000", 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.276512+0000", 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.276514+0000", 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7400000000000001e-06 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.276559+0000", 2024-04-18T23:12:10.796 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.5278999999999998e-05 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.276625+0000", 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6284000000000004e-05 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.276654+0000", 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8898999999999999e-05 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.373517+0000", 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.096863006000000001 2024-04-18T23:12:10.797 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.373548+0000", 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0599000000000003e-05 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.474548+0000", 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10100023299999999 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.474581+0000", 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2361999999999999e-05 2024-04-18T23:12:10.798 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.cs5 100/31 ECSubWrite(tid=108, reqid=client.4568.0:943, at_version=0'0, trim_to=0'0, roll_forward_to=100'69))", 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:07.137781+0000", 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 3.5797012869999998, 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.201701979, 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.799 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.137781+0000", 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.137781+0000", 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.137783+0000", 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2750000000000001e-06 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:10.800 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.137797+0000", 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4175999999999999e-05 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.137814+0000", 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7282000000000002e-05 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.137825+0000", 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0679000000000001e-05 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:10.801 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.338138+0000", 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.200313452 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.338158+0000", 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0146e-05 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.339426+0000", 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001267712 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:10.802 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:10.803 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.339483+0000", 2024-04-18T23:12:10.803 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.7256999999999997e-05 2024-04-18T23:12:10.803 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.803 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.803 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.803 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:10.803 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:10.803 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:10.805 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:12:10.805 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:10.804+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:10.905 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:10.902+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:10.927 INFO:tasks.rados.rados.0.smithi142.stdout:399: finishing write tid 6 to smithi14230322-5 2024-04-18T23:12:10.928 INFO:tasks.rados.rados.0.smithi142.stdout:399: oid 5 updating version 0 to 183 2024-04-18T23:12:10.928 INFO:tasks.rados.rados.0.smithi142.stdout:399: oid 5 updating version 183 to 187 2024-04-18T23:12:10.928 INFO:tasks.rados.rados.0.smithi142.stdout:399: oid 5 version 187 is already newer than 186 2024-04-18T23:12:10.928 INFO:tasks.rados.rados.0.smithi142.stdout:399: oid 5 version 187 is already newer than 184 2024-04-18T23:12:10.928 INFO:tasks.rados.rados.0.smithi142.stdout:399: oid 5 version 187 is already newer than 185 2024-04-18T23:12:10.928 INFO:tasks.rados.rados.0.smithi142.stdout:update_object_version oid 5 v 187 (ObjNum 142 snap 38 seq_num 142) dirty exists 2024-04-18T23:12:10.928 INFO:tasks.rados.rados.0.smithi142.stdout:399: left oid 5 (ObjNum 142 snap 38 seq_num 142) 2024-04-18T23:12:10.928 INFO:tasks.rados.rados.0.smithi142.stdout:399: done (0 left) 2024-04-18T23:12:10.937 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:12:10.937 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:12:10.937 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:10.938 INFO:tasks.rados.rados.0.smithi142.stderr:0 errors. 2024-04-18T23:12:10.938 INFO:tasks.rados.rados.0.smithi142.stderr: 2024-04-18T23:12:10.945 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T23:12:10.951 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:12:11.006 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:11.007+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:11.070 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:12:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:12:11.070 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:12:11.070 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:11.084 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:12:11.131 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:11.130+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:11.207 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:11.207+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:11.209 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:12:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:12:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:12:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 61/31 ECSubWrite(tid=186, reqid=osd.4.0:186, at_version=61'68, trim_to=0'0, roll_forward_to=61'66))", 2024-04-18T23:12:11.209 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.252802123999999, 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201499973, 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.210 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956659+0000", 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956661+0000", 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7370000000000001e-06 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.211 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956723+0000", 2024-04-18T23:12:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2564999999999995e-05 2024-04-18T23:12:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:26.956762+0000", 2024-04-18T23:12:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8711999999999998e-05 2024-04-18T23:12:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.212 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.056877+0000", 2024-04-18T23:12:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10011492900000001 2024-04-18T23:12:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.213 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.057020+0000", 2024-04-18T23:12:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142662 2024-04-18T23:12:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.057046+0000", 2024-04-18T23:12:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6486e-05 2024-04-18T23:12:11.214 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.158121+0000", 2024-04-18T23:12:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10107499 2024-04-18T23:12:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.215 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.158159+0000", 2024-04-18T23:12:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7892000000000002e-05 2024-04-18T23:12:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.216 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 61/31 ECSubWrite(tid=45, reqid=osd.2.0:45, at_version=61'33, trim_to=0'0, roll_forward_to=61'31))", 2024-04-18T23:12:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:12:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 43.863874312, 2024-04-18T23:12:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20243294100000001, 2024-04-18T23:12:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.217 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:12:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.218 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345587+0000", 2024-04-18T23:12:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.219 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345589+0000", 2024-04-18T23:12:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7540000000000001e-06 2024-04-18T23:12:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345643+0000", 2024-04-18T23:12:11.220 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4707999999999999e-05 2024-04-18T23:12:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345662+0000", 2024-04-18T23:12:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8870000000000001e-05 2024-04-18T23:12:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.221 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.345675+0000", 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2482e-05 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.545813+0000", 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20013862499999999 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.545829+0000", 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5201999999999999e-05 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.222 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.547933+0000", 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021040389999999998 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.548020+0000", 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7261000000000004e-05 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.223 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 70/31 ECSubWrite(tid=57, reqid=client.4568.0:551, at_version=70'43, trim_to=0'0, roll_forward_to=69'42))", 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.565666+0000", 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.643795427000001, 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202564679, 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.224 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565666+0000", 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565666+0000", 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565667+0000", 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2699999999999998e-07 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.225 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565871+0000", 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020396499999999999 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.565899+0000", 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8572e-05 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.666000+0000", 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100101172 2024-04-18T23:12:11.226 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.766291+0000", 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100290465 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.766307+0000", 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5979999999999999e-05 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.768156+0000", 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0018494799999999999 2024-04-18T23:12:11.227 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.768230+0000", 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4218e-05 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es2 70/31 ECSubWrite(tid=61, reqid=client.4568.0:554, at_version=0'0, trim_to=0'0, roll_forward_to=70'46))", 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.082659+0000", 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.126802200999997, 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20138941299999999, 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.228 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082659+0000", 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082659+0000", 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082661+0000", 2024-04-18T23:12:11.229 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.694e-06 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082682+0000", 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1231999999999999e-05 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.082712+0000", 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0273e-05 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.230 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.182872+0000", 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100159647 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.183044+0000", 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000172356 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.231 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.183071+0000", 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7097000000000002e-05 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.283975+0000", 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100903359 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.233 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.284048+0000", 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3755000000000001e-05 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 77/31 ECSubWrite(tid=247, reqid=client.4568.0:607, at_version=77'85, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.821380+0000", 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.388080820999999, 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20502453600000001, 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.234 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821380+0000", 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821380+0000", 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821382+0000", 2024-04-18T23:12:11.235 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.973e-06 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821601+0000", 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000219053 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.821630+0000", 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8651999999999999e-05 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921792+0000", 2024-04-18T23:12:11.236 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10016209 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921968+0000", 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017578900000000001 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921988+0000", 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0029000000000001e-05 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.026322+0000", 2024-04-18T23:12:11.237 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10433400499999999 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.026405+0000", 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2945e-05 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 77/31 ECSubWrite(tid=248, reqid=client.4568.0:608, at_version=77'86, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.921849+0000", 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.287612487000001, 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20508053000000001, 2024-04-18T23:12:11.238 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921849+0000", 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921849+0000", 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.239 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921850+0000", 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0839999999999999e-06 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.921982+0000", 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000132464 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922007+0000", 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5089e-05 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.240 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922021+0000", 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3113e-05 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.923052+0000", 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0010313989999999999 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.923081+0000", 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9125e-05 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.241 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126891+0000", 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.203809765 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126929+0000", 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8491000000000001e-05 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 77/31 ECSubWrite(tid=249, reqid=client.4568.0:609, at_version=77'87, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.922042+0000", 2024-04-18T23:12:11.242 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.287419449000001, 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20493312899999999, 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922042+0000", 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922042+0000", 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.243 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922042+0000", 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3399999999999999e-07 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922064+0000", 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1611e-05 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922076+0000", 2024-04-18T23:12:11.244 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1732999999999999e-05 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922085+0000", 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0259999999999996e-06 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.924159+0000", 2024-04-18T23:12:11.245 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020747249999999999 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.924190+0000", 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0151e-05 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126939+0000", 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20274966799999999 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.246 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126975+0000", 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5680999999999999e-05 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.bs2 77/31 ECSubRead(tid=254, to_read={3:d447fef0:::smithi14230322-49:16=0,475136,0}, subchunks={3:d447fef0:::smithi14230322-49:16=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.922199+0000", 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.287262405, 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.204785145, 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.247 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922199+0000", 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922199+0000", 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922199+0000", 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2000000000000001e-07 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.248 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922204+0000", 2024-04-18T23:12:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2290000000000002e-06 2024-04-18T23:12:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922219+0000", 2024-04-18T23:12:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4676e-05 2024-04-18T23:12:11.249 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.922234+0000", 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4763000000000001e-05 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126655+0000", 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20442090399999999 2024-04-18T23:12:11.250 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.126984+0000", 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000329253 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 81/72 ECSubWrite(tid=187, reqid=osd.14.0:187, at_version=81'90, trim_to=0'0, roll_forward_to=80'88))", 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:47.647535+0000", 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.561926372999999, 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201612076, 2024-04-18T23:12:11.251 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647535+0000", 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647535+0000", 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.252 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647536+0000", 2024-04-18T23:12:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2399999999999997e-07 2024-04-18T23:12:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647553+0000", 2024-04-18T23:12:11.253 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7581999999999999e-05 2024-04-18T23:12:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.647565+0000", 2024-04-18T23:12:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1487e-05 2024-04-18T23:12:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.254 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.747635+0000", 2024-04-18T23:12:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100070545 2024-04-18T23:12:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.747765+0000", 2024-04-18T23:12:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013010499999999999 2024-04-18T23:12:11.255 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.747782+0000", 2024-04-18T23:12:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6935999999999998e-05 2024-04-18T23:12:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.256 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.849112+0000", 2024-04-18T23:12:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10133017900000001 2024-04-18T23:12:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.849147+0000", 2024-04-18T23:12:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4517999999999997e-05 2024-04-18T23:12:11.257 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 81/31 ECSubWrite(tid=299, reqid=osd.4.0:299, at_version=81'116, trim_to=0'0, roll_forward_to=80'112))", 2024-04-18T23:12:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:47.737322+0000", 2024-04-18T23:12:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.472139488, 2024-04-18T23:12:11.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201559339, 2024-04-18T23:12:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737322+0000", 2024-04-18T23:12:11.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737322+0000", 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737322+0000", 2024-04-18T23:12:11.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9999999999999997e-07 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737355+0000", 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2598000000000002e-05 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.737368+0000", 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2702e-05 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.837446+0000", 2024-04-18T23:12:11.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100078242 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.937649+0000", 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100203006 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.937683+0000", 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4464000000000001e-05 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.938838+0000", 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001154293 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:47.938881+0000", 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3433999999999998e-05 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.6s2 81/72 ECSubWrite(tid=49, reqid=osd.0.0:48, at_version=0'0, trim_to=0'0, roll_forward_to=81'36))", 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.042478+0000", 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.166983193, 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20112732899999999, 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.263 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042478+0000", 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042478+0000", 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042479+0000", 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1969999999999999e-06 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042496+0000", 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6589e-05 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.042548+0000", 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2468e-05 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.142626+0000", 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100077679 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.142727+0000", 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00010097799999999999 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.142744+0000", 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6801999999999999e-05 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.266 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.243552+0000", 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10080777 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.243605+0000", 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.3845999999999999e-05 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 82/72 ECSubWrite(tid=194, reqid=client.4568.0:674, at_version=82'98, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.661633+0000", 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.547828108000001, 2024-04-18T23:12:11.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30563710599999999, 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.661633+0000", 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.661633+0000", 2024-04-18T23:12:11.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.661635+0000", 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.662171+0000", 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00053610699999999997 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.662200+0000", 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8795000000000001e-05 2024-04-18T23:12:11.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.662222+0000", 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2362999999999999e-05 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762467+0000", 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100244747 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762499+0000", 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1915000000000002e-05 2024-04-18T23:12:11.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.967208+0000", 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20470896599999999 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.967270+0000", 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2663000000000006e-05 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.271 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 82/72 ECSubWrite(tid=195, reqid=client.4568.0:675, at_version=82'99, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.760858+0000", 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.448602895000001, 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20646499700000001, 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.760858+0000", 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.272 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.760858+0000", 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.760861+0000", 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5059999999999998e-06 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.761310+0000", 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00044913199999999998 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.761341+0000", 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1362000000000002e-05 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.761357+0000", 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5792999999999999e-05 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.764635+0000", 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0032775450000000002 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.764664+0000", 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9218999999999999e-05 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.967284+0000", 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20261981500000001 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.967323+0000", 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9625000000000001e-05 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.275 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 82/72 ECSubWrite(tid=196, reqid=client.4568.0:676, at_version=82'100, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.762672+0000", 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.446789054, 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20774319099999999, 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762672+0000", 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762672+0000", 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762674+0000", 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1670000000000002e-06 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762829+0000", 2024-04-18T23:12:11.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000155081 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762858+0000", 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8119e-05 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.762873+0000", 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5559000000000001e-05 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.966297+0000", 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20342418200000001 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.966314+0000", 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6609e-05 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.970345+0000", 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0040308879999999998 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.970415+0000", 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.0586000000000005e-05 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.4s3 82/31 ECSubWrite(tid=115, reqid=osd.8.0:115, at_version=82'72, trim_to=0'0, roll_forward_to=81'70))", 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.777459+0000", 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.432002126, 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20149748300000001, 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.280 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777459+0000", 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777459+0000", 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777460+0000", 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9999999999999996e-07 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777511+0000", 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.0643000000000003e-05 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.777540+0000", 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9419000000000001e-05 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.877642+0000", 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100102052 2024-04-18T23:12:11.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.877807+0000", 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016472399999999999 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.877826+0000", 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8958999999999999e-05 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.978871+0000", 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10104500399999999 2024-04-18T23:12:11.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.978957+0000", 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.5882000000000002e-05 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.9s1 87/72 ECSubWrite(tid=230, reqid=osd.14.0:230, at_version=87'122, trim_to=0'0, roll_forward_to=87'120))", 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:53.983920+0000", 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.225541485000001, 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201235989, 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.284 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983920+0000", 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983920+0000", 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983921+0000", 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5799999999999999e-06 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983971+0000", 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9509999999999999e-05 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:53.983996+0000", 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4819000000000001e-05 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.084083+0000", 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100087079 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.084223+0000", 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014055 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.084252+0000", 2024-04-18T23:12:11.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8286e-05 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.185099+0000", 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100847145 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.185156+0000", 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.702e-05 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs2 87/31 ECSubWrite(tid=377, reqid=osd.4.0:374, at_version=0'0, trim_to=0'0, roll_forward_to=87'149))", 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:54.146955+0000", 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.062505990999998, 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.203462847, 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.146955+0000", 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.146955+0000", 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.146957+0000", 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7859999999999999e-06 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.146978+0000", 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0933000000000001e-05 2024-04-18T23:12:11.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.147007+0000", 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9288000000000001e-05 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.247164+0000", 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100157156 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.347422+0000", 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10025711900000001 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.347455+0000", 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3575999999999998e-05 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.350346+0000", 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028906890000000001 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.350418+0000", 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.2299999999999996e-05 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.292 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.2s5 94/67 ECSubWrite(tid=265, reqid=osd.13.0:264, at_version=0'0, trim_to=0'0, roll_forward_to=94'85))", 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:12:01.124166+0000", 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 10.085295532, 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20126012900000001, 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.124166+0000", 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.124166+0000", 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.124168+0000", 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.973e-06 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.124190+0000", 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2580000000000001e-05 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.124221+0000", 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1214999999999998e-05 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.224381+0000", 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100159333 2024-04-18T23:12:11.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.224546+0000", 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016472399999999999 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.224567+0000", 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1472999999999999e-05 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.325334+0000", 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100766721 2024-04-18T23:12:11.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:01.325426+0000", 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2109999999999997e-05 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.bs2 100/31 ECSubRead(tid=468, to_read={3:d447fef0:::smithi14230322-49:head=0,1490944,0}, subchunks={3:d447fef0:::smithi14230322-49:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:12:06.783081+0000", 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 4.4263804560000004, 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20244384000000001, 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:06.783081+0000", 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:06.783081+0000", 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:06.783082+0000", 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1570000000000001e-06 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:06.783094+0000", 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2126e-05 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:06.783116+0000", 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1603e-05 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:06.783139+0000", 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3005000000000001e-05 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:06.985140+0000", 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20200131499999999 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:06.985525+0000", 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00038463400000000002 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.300 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.as2 103/31 ECSubWrite(tid=58, reqid=client.4568.0:1020, at_version=103'34, trim_to=0'0, roll_forward_to=100'30))", 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:12:10.222939+0000", 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 0.98652197699999999, 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.203076232, 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.222939+0000", 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.222939+0000", 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.222941+0000", 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7430000000000001e-06 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.223094+0000", 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000153249 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.223124+0000", 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9998999999999999e-05 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.223140+0000", 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5362999999999998e-05 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.323731+0000", 2024-04-18T23:12:11.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10059151700000001 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.323763+0000", 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1819000000000001e-05 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.425942+0000", 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10217918099999999 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:12:10.426015+0000", 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.3361000000000005e-05 2024-04-18T23:12:11.304 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.305 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.305 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:11.306 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:12:11.307 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":103,"fsid":"b8d0ffd9-6668-442b-a039-a24cea734f3e","created":"2024-04-18T23:10:28.548637+0000","modified":"2024-04-18T23:12:09.512339+0000","last_up_change":"2024-04-18T23:11:32.525078+0000","last_in_change":"2024-04-18T23:10:33.002063+0000","flags":"nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5804032,"flags_set":["nodeep-scrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T23:10:41.349753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T23:10:43.462157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"unique_pool_0","create_time":"2024-04-18T23:10:52.461655+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":3,"size":7,"min_size":5,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":16,"pg_placement_num":16,"pg_placement_num_target":16,"pg_num_target":16,"pg_num_pending":16,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"103","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":39,"snap_epoch":103,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"shecprofile","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":16384,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rados":{}}}],"osds":[{"osd":0,"uuid":"3af51a22-0d99-447f-8e09-ef2a2077dcba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6803","nonce":169883533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6805","nonce":169883533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6817","nonce":169883533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6808","nonce":169883533}]},"public_addr":"172.21.15.73:6803/169883533","cluster_addr":"172.21.15.73:6805/169883533","heartbeat_back_addr":"172.21.15.73:6817/169883533","heartbeat_front_addr":"172.21.15.73:6808/169883533","state":["exists","up"]},{"osd":1,"uuid":"f50deb0a-c98a-45ec-9ee3-be20aff17eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6809","nonce":3147085605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6811","nonce":3147085605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6816","nonce":3147085605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6813","nonce":3147085605}]},"public_addr":"172.21.15.120:6809/3147085605","cluster_addr":"172.21.15.120:6811/3147085605","heartbeat_back_addr":"172.21.15.120:6816/3147085605","heartbeat_front_addr":"172.21.15.120:6813/3147085605","state":["exists","up"]},{"osd":2,"uuid":"b6556b6f-9ef8-462d-ae9f-b59713dec207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6809","nonce":1984461220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6819","nonce":1984461220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6831","nonce":1984461220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6823","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6827","nonce":1984461220}]},"public_addr":"172.21.15.136:6809/1984461220","cluster_addr":"172.21.15.136:6819/1984461220","heartbeat_back_addr":"172.21.15.136:6831/1984461220","heartbeat_front_addr":"172.21.15.136:6827/1984461220","state":["exists","up"]},{"osd":3,"uuid":"2448b4d0-af77-4951-bcf3-f2752e7d13b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6817","nonce":2108797520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6819","nonce":2108797520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6823","nonce":2108797520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6821","nonce":2108797520}]},"public_addr":"172.21.15.142:6817/2108797520","cluster_addr":"172.21.15.142:6819/2108797520","heartbeat_back_addr":"172.21.15.142:6823/2108797520","heartbeat_front_addr":"172.21.15.142:6821/2108797520","state":["exists","up"]},{"osd":4,"uuid":"d8ae2413-5474-40ca-b256-04b012595463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6812","nonce":1768990367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6820","nonce":1768990367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6831","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6833","nonce":1768990367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6823","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6827","nonce":1768990367}]},"public_addr":"172.21.15.73:6812/1768990367","cluster_addr":"172.21.15.73:6820/1768990367","heartbeat_back_addr":"172.21.15.73:6833/1768990367","heartbeat_front_addr":"172.21.15.73:6827/1768990367","state":["exists","up"]},{"osd":5,"uuid":"3515f9b0-470c-4db3-b456-a3985a36bf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6801","nonce":4018604021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6803","nonce":4018604021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6807","nonce":4018604021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6805","nonce":4018604021}]},"public_addr":"172.21.15.120:6801/4018604021","cluster_addr":"172.21.15.120:6803/4018604021","heartbeat_back_addr":"172.21.15.120:6807/4018604021","heartbeat_front_addr":"172.21.15.120:6805/4018604021","state":["exists","up"]},{"osd":6,"uuid":"12d2f5e8-9eaa-4c21-a250-1db8c4e641cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6803","nonce":2261503449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6811","nonce":2261503449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6826","nonce":2261503449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6818","nonce":2261503449}]},"public_addr":"172.21.15.136:6803/2261503449","cluster_addr":"172.21.15.136:6811/2261503449","heartbeat_back_addr":"172.21.15.136:6826/2261503449","heartbeat_front_addr":"172.21.15.136:6818/2261503449","state":["exists","up"]},{"osd":7,"uuid":"c9e5e983-106f-4d01-a153-cdf7d656a091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6804","nonce":1398734507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6807","nonce":1398734507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6814","nonce":1398734507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6810","nonce":1398734507}]},"public_addr":"172.21.15.142:6804/1398734507","cluster_addr":"172.21.15.142:6807/1398734507","heartbeat_back_addr":"172.21.15.142:6814/1398734507","heartbeat_front_addr":"172.21.15.142:6810/1398734507","state":["exists","up"]},{"osd":8,"uuid":"a711d0fa-f5a7-4146-b280-8ee969afbb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6815","nonce":786826444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6819","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6821","nonce":786826444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6829","nonce":786826444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6826","nonce":786826444}]},"public_addr":"172.21.15.73:6815/786826444","cluster_addr":"172.21.15.73:6821/786826444","heartbeat_back_addr":"172.21.15.73:6829/786826444","heartbeat_front_addr":"172.21.15.73:6826/786826444","state":["exists","up"]},{"osd":9,"uuid":"914af14e-f90d-42de-918a-d925f50ccfb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6817","nonce":2287493993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6819","nonce":2287493993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6823","nonce":2287493993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6821","nonce":2287493993}]},"public_addr":"172.21.15.120:6817/2287493993","cluster_addr":"172.21.15.120:6819/2287493993","heartbeat_back_addr":"172.21.15.120:6823/2287493993","heartbeat_front_addr":"172.21.15.120:6821/2287493993","state":["exists","up"]},{"osd":10,"uuid":"6ad314a3-6c86-46fd-b1a6-d20911da6d24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6802","nonce":1419708480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6808","nonce":1419708480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6825","nonce":1419708480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6817","nonce":1419708480}]},"public_addr":"172.21.15.136:6802/1419708480","cluster_addr":"172.21.15.136:6808/1419708480","heartbeat_back_addr":"172.21.15.136:6825/1419708480","heartbeat_front_addr":"172.21.15.136:6817/1419708480","state":["exists","up"]},{"osd":11,"uuid":"2ea8b70d-e122-4c29-8742-18d341fd8668","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6825","nonce":2871475141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6827","nonce":2871475141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6831","nonce":2871475141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6829","nonce":2871475141}]},"public_addr":"172.21.15.142:6825/2871475141","cluster_addr":"172.21.15.142:6827/2871475141","heartbeat_back_addr":"172.21.15.142:6831/2871475141","heartbeat_front_addr":"172.21.15.142:6829/2871475141","state":["exists","up"]},{"osd":12,"uuid":"24086ac0-a92d-400b-9b3d-f48992dc8679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6810","nonce":3885787537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6818","nonce":3885787537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6832","nonce":3885787537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6825","nonce":3885787537}]},"public_addr":"172.21.15.73:6810/3885787537","cluster_addr":"172.21.15.73:6818/3885787537","heartbeat_back_addr":"172.21.15.73:6832/3885787537","heartbeat_front_addr":"172.21.15.73:6825/3885787537","state":["exists","up"]},{"osd":13,"uuid":"da98eea5-e373-4325-adda-75f747ad6a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6825","nonce":2634549424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6827","nonce":2634549424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6831","nonce":2634549424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6829","nonce":2634549424}]},"public_addr":"172.21.15.120:6825/2634549424","cluster_addr":"172.21.15.120:6827/2634549424","heartbeat_back_addr":"172.21.15.120:6831/2634549424","heartbeat_front_addr":"172.21.15.120:6829/2634549424","state":["exists","up"]},{"osd":14,"uuid":"d03df2d4-8d1c-4c2a-9719-d64ad780acc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6810","nonce":381469963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6816","nonce":381469963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6829","nonce":381469963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6824","nonce":381469963}]},"public_addr":"172.21.15.136:6810/381469963","cluster_addr":"172.21.15.136:6816/381469963","heartbeat_back_addr":"172.21.15.136:6829/381469963","heartbeat_front_addr":"172.21.15.136:6824/381469963","state":["exists","up"]},{"osd":15,"uuid":"7aa92378-d336-41be-85f1-2eb582f54947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":22,"last_clean_end":48,"up_from":67,"up_thru":72,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6801","nonce":1087448681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6806","nonce":1087448681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6815","nonce":1087448681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6811","nonce":1087448681}]},"public_addr":"172.21.15.142:6801/1087448681","cluster_addr":"172.21.15.142:6806/1087448681","heartbeat_back_addr":"172.21.15.142:6815/1087448681","heartbeat_front_addr":"172.21.15.142:6811/1087448681","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:57.350505+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:40.964607+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:07.537155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:58.962904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:43.708328+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:41.635275+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:06.821551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:05.573559+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:59.025889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:43.868185+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:49.874834+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:00.497097+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:59.226387+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:52.871228+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:39.826822+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:02.044564+0000","dead_epoch":49}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[{"pool":3,"snaps":[{"begin":35,"length":1},{"begin":37,"length":1}]}],"new_removed_snaps":[{"pool":3,"snaps":[{"begin":37,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T23:12:11.307 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:12:11.308 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:12:11.351 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:11.350+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:11.410 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:11.408+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:11.481 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:12:11.481 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:12:11.481 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:11.494 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:12:11.511 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:11.511+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:11.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T23:12:11.612 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:11.610+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:11.616 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:12:11.617 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:12:11.618 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:12:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:12:11.618 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:12:11.618 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:11.632 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:12:11.634 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":133,"stamp":"2024-04-18T23:12:11.290945+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":136512032,"num_objects":131,"num_object_clones":79,"num_object_copies":907,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":131,"num_whiteouts":5,"num_read":584,"num_read_kb":146751,"num_write":1044,"num_write_kb":156388,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1588,"ondisk_log_size":1588,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":735808,"kb_used_data":247828,"kb_used_omap":35,"kb_used_meta":487708,"kb_avail":1509213632,"statfs":{"total":1546188226560,"available":1545434759168,"internally_reserved":0,"allocated":253775872,"data_stored":369659994,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":499413633},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,5,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":28082176,"num_objects":12,"num_object_clones":12,"num_object_copies":84,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":-1,"num_read":14,"num_read_kb":7018,"num_write":57,"num_write_kb":27424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.723465"},"pg_stats":[{"pgid":"3.f","version":"80'19","reported_seq":338,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.583831+0000","last_change":"2024-04-18T23:12:05.584453+0000","last_active":"2024-04-18T23:12:05.583831+0000","last_peered":"2024-04-18T23:12:05.583831+0000","last_clean":"2024-04-18T23:12:05.583831+0000","last_became_active":"2024-04-18T23:10:54.696778+0000","last_became_peered":"2024-04-18T23:10:54.696778+0000","last_unstale":"2024-04-18T23:12:05.583831+0000","last_undegraded":"2024-04-18T23:12:05.583831+0000","last_fullsized":"2024-04-18T23:12:05.583831+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096931999999999997,"stat_sum":{"num_bytes":1933312,"num_objects":1,"num_object_clones":0,"num_object_copies":7,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":19110,"num_write":6,"num_write_kb":1888,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.e","version":"87'84","reported_seq":402,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.652030+0000","last_change":"2024-04-18T23:12:05.652610+0000","last_active":"2024-04-18T23:12:05.652030+0000","last_peered":"2024-04-18T23:12:05.652030+0000","last_clean":"2024-04-18T23:12:05.652030+0000","last_became_active":"2024-04-18T23:10:54.918517+0000","last_became_peered":"2024-04-18T23:10:54.918517+0000","last_unstale":"2024-04-18T23:12:05.652030+0000","last_undegraded":"2024-04-18T23:12:05.652030+0000","last_fullsized":"2024-04-18T23:12:05.652030+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'84","last_scrub_stamp":"2024-04-18T23:12:01.429717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.429717+0000","objects_scrubbed":6,"log_size":84,"log_dups_size":0,"ondisk_log_size":84,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:06.684408+0000","scrub_duration":815,"objects_trimmed":0,"snaptrim_duration":0.00088648799999999995,"stat_sum":{"num_bytes":6995968,"num_objects":6,"num_object_clones":3,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":23,"num_read_kb":16277,"num_write":54,"num_write_kb":8016,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.d","version":"100'167","reported_seq":499,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:07.085218+0000","last_change":"2024-04-18T23:12:06.088542+0000","last_active":"2024-04-18T23:12:07.085218+0000","last_peered":"2024-04-18T23:12:07.085218+0000","last_clean":"2024-04-18T23:12:07.085218+0000","last_became_active":"2024-04-18T23:10:54.712663+0000","last_became_peered":"2024-04-18T23:10:54.712663+0000","last_unstale":"2024-04-18T23:12:07.085218+0000","last_undegraded":"2024-04-18T23:12:07.085218+0000","last_fullsized":"2024-04-18T23:12:07.085218+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'148","last_scrub_stamp":"2024-04-18T23:12:03.759433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:03.759433+0000","objects_scrubbed":13,"log_size":167,"log_dups_size":0,"ondisk_log_size":167,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:24.948937+0000","scrub_duration":319,"objects_trimmed":0,"snaptrim_duration":0.31019849700000002,"stat_sum":{"num_bytes":18448384,"num_objects":15,"num_object_clones":10,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":58,"num_read_kb":12309,"num_write":118,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.c","version":"99'67","reported_seq":380,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.788916+0000","last_change":"2024-04-18T23:12:05.790158+0000","last_active":"2024-04-18T23:12:05.788916+0000","last_peered":"2024-04-18T23:12:05.788916+0000","last_clean":"2024-04-18T23:12:05.788916+0000","last_became_active":"2024-04-18T23:10:54.717053+0000","last_became_peered":"2024-04-18T23:10:54.717053+0000","last_unstale":"2024-04-18T23:12:05.788916+0000","last_undegraded":"2024-04-18T23:12:05.788916+0000","last_fullsized":"2024-04-18T23:12:05.788916+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:15.192684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.106631478,"stat_sum":{"num_bytes":9977856,"num_objects":7,"num_object_clones":5,"num_object_copies":49,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":22,"num_read_kb":8759,"num_write":42,"num_write_kb":10400,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.b","version":"99'185","reported_seq":504,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.355644+0000","last_change":"2024-04-18T23:12:06.360059+0000","last_active":"2024-04-18T23:12:06.355644+0000","last_peered":"2024-04-18T23:12:06.355644+0000","last_clean":"2024-04-18T23:12:06.355644+0000","last_became_active":"2024-04-18T23:10:54.808152+0000","last_became_peered":"2024-04-18T23:10:54.808152+0000","last_unstale":"2024-04-18T23:12:06.355644+0000","last_undegraded":"2024-04-18T23:12:06.355644+0000","last_fullsized":"2024-04-18T23:12:06.355644+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'175","last_scrub_stamp":"2024-04-18T23:12:03.002648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:03.002648+0000","objects_scrubbed":14,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:12.029331+0000","scrub_duration":819,"objects_trimmed":0,"snaptrim_duration":0.52115570700000002,"stat_sum":{"num_bytes":26034176,"num_objects":16,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":41,"num_read_kb":26756,"num_write":83,"num_write_kb":25696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.a","version":"99'28","reported_seq":357,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.683002+0000","last_change":"2024-04-18T23:12:05.684609+0000","last_active":"2024-04-18T23:12:05.683002+0000","last_peered":"2024-04-18T23:12:05.683002+0000","last_clean":"2024-04-18T23:12:05.683002+0000","last_became_active":"2024-04-18T23:10:54.764999+0000","last_became_peered":"2024-04-18T23:10:54.764999+0000","last_unstale":"2024-04-18T23:12:05.683002+0000","last_undegraded":"2024-04-18T23:12:05.683002+0000","last_fullsized":"2024-04-18T23:12:05.683002+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.106914625,"stat_sum":{"num_bytes":720896,"num_objects":4,"num_object_clones":2,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":1,"num_read":28,"num_read_kb":4232,"num_write":7,"num_write_kb":624,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.9","version":"99'143","reported_seq":506,"reported_epoch":99,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:12:06.093010+0000","last_change":"2024-04-18T23:12:05.684570+0000","last_active":"2024-04-18T23:12:06.093010+0000","last_peered":"2024-04-18T23:12:06.093010+0000","last_clean":"2024-04-18T23:12:06.093010+0000","last_became_active":"2024-04-18T23:11:39.726824+0000","last_became_peered":"2024-04-18T23:11:39.726824+0000","last_unstale":"2024-04-18T23:12:06.093010+0000","last_undegraded":"2024-04-18T23:12:06.093010+0000","last_fullsized":"2024-04-18T23:12:06.093010+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'124","last_scrub_stamp":"2024-04-18T23:11:59.890703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:11:59.890703+0000","objects_scrubbed":13,"log_size":143,"log_dups_size":0,"ondisk_log_size":143,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:16.200845+0000","scrub_duration":619,"objects_trimmed":0,"snaptrim_duration":0.00090971900000000002,"stat_sum":{"num_bytes":5767168,"num_objects":17,"num_object_clones":11,"num_object_copies":119,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":1,"num_read":56,"num_read_kb":10769,"num_write":108,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2867200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"3.8","version":"87'61","reported_seq":411,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.691590+0000","last_change":"2024-04-18T23:12:05.792408+0000","last_active":"2024-04-18T23:12:05.691590+0000","last_peered":"2024-04-18T23:12:05.691590+0000","last_clean":"2024-04-18T23:12:05.691590+0000","last_became_active":"2024-04-18T23:11:37.876250+0000","last_became_peered":"2024-04-18T23:11:37.876250+0000","last_unstale":"2024-04-18T23:12:05.691590+0000","last_undegraded":"2024-04-18T23:12:05.691590+0000","last_fullsized":"2024-04-18T23:12:05.691590+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'61","last_scrub_stamp":"2024-04-18T23:12:02.908053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:02.908053+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:10.164274+0000","scrub_duration":515,"objects_trimmed":0,"snaptrim_duration":0.101181459,"stat_sum":{"num_bytes":1949696,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":8055,"num_write":54,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"1.0","version":"33'157","reported_seq":366,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.660423+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:12:05.660423+0000","last_peered":"2024-04-18T23:12:05.660423+0000","last_clean":"2024-04-18T23:12:05.660423+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:12:05.660423+0000","last_undegraded":"2024-04-18T23:12:05.660423+0000","last_fullsized":"2024-04-18T23:12:05.660423+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":169,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.690884+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:12:05.690884+0000","last_peered":"2024-04-18T23:12:05.690884+0000","last_clean":"2024-04-18T23:12:05.690884+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:12:05.690884+0000","last_undegraded":"2024-04-18T23:12:05.690884+0000","last_fullsized":"2024-04-18T23:12:05.690884+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"99'93","reported_seq":429,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.865456+0000","last_change":"2024-04-18T23:12:05.867245+0000","last_active":"2024-04-18T23:12:05.865456+0000","last_peered":"2024-04-18T23:12:05.865456+0000","last_clean":"2024-04-18T23:12:05.865456+0000","last_became_active":"2024-04-18T23:11:33.818951+0000","last_became_peered":"2024-04-18T23:11:33.818951+0000","last_unstale":"2024-04-18T23:12:05.865456+0000","last_undegraded":"2024-04-18T23:12:05.865456+0000","last_fullsized":"2024-04-18T23:12:05.865456+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":93,"log_dups_size":0,"ondisk_log_size":93,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20639465200000001,"stat_sum":{"num_bytes":12517376,"num_objects":10,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":28,"num_read_kb":6012,"num_write":40,"num_write_kb":12528,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.0","version":"0'0","reported_seq":164,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.577324+0000","last_change":"2024-04-18T23:12:01.049893+0000","last_active":"2024-04-18T23:12:05.577324+0000","last_peered":"2024-04-18T23:12:05.577324+0000","last_clean":"2024-04-18T23:12:05.577324+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:12:05.577324+0000","last_undegraded":"2024-04-18T23:12:05.577324+0000","last_fullsized":"2024-04-18T23:12:05.577324+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:01.049599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.049599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:09.776200+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"88'10","reported_seq":322,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.741453+0000","last_change":"2024-04-18T23:12:05.741688+0000","last_active":"2024-04-18T23:12:05.741453+0000","last_peered":"2024-04-18T23:12:05.741453+0000","last_clean":"2024-04-18T23:12:05.741453+0000","last_became_active":"2024-04-18T23:10:54.799189+0000","last_became_peered":"2024-04-18T23:10:54.799189+0000","last_unstale":"2024-04-18T23:12:05.741453+0000","last_undegraded":"2024-04-18T23:12:05.741453+0000","last_fullsized":"2024-04-18T23:12:05.741453+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.356352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00035962899999999998,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":156,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.660548+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:12:05.660548+0000","last_peered":"2024-04-18T23:12:05.660548+0000","last_clean":"2024-04-18T23:12:05.660548+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:12:05.660548+0000","last_undegraded":"2024-04-18T23:12:05.660548+0000","last_fullsized":"2024-04-18T23:12:05.660548+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:14.353599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"87'44","reported_seq":389,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.685253+0000","last_change":"2024-04-18T23:12:05.685253+0000","last_active":"2024-04-18T23:12:05.685253+0000","last_peered":"2024-04-18T23:12:05.685253+0000","last_clean":"2024-04-18T23:12:05.685253+0000","last_became_active":"2024-04-18T23:11:38.818022+0000","last_became_peered":"2024-04-18T23:11:38.818022+0000","last_unstale":"2024-04-18T23:12:05.685253+0000","last_undegraded":"2024-04-18T23:12:05.685253+0000","last_fullsized":"2024-04-18T23:12:05.685253+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.10090613700000001,"stat_sum":{"num_bytes":6438912,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":16,"num_read_kb":2792,"num_write":20,"num_write_kb":6288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":7716864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.2","version":"0'0","reported_seq":155,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.577536+0000","last_change":"2024-04-18T23:10:45.496538+0000","last_active":"2024-04-18T23:12:05.577536+0000","last_peered":"2024-04-18T23:12:05.577536+0000","last_clean":"2024-04-18T23:12:05.577536+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:12:05.577536+0000","last_undegraded":"2024-04-18T23:12:05.577536+0000","last_fullsized":"2024-04-18T23:12:05.577536+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:12.994274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":324,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.661804+0000","last_change":"2024-04-18T23:12:05.662471+0000","last_active":"2024-04-18T23:12:05.661804+0000","last_peered":"2024-04-18T23:12:05.661804+0000","last_clean":"2024-04-18T23:12:05.661804+0000","last_became_active":"2024-04-18T23:11:33.713370+0000","last_became_peered":"2024-04-18T23:11:33.713370+0000","last_unstale":"2024-04-18T23:12:05.661804+0000","last_undegraded":"2024-04-18T23:12:05.661804+0000","last_fullsized":"2024-04-18T23:12:05.661804+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:10.383500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001186725,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"3.5","version":"100'129","reported_seq":462,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.571515+0000","last_change":"2024-04-18T23:12:05.669892+0000","last_active":"2024-04-18T23:12:06.571515+0000","last_peered":"2024-04-18T23:12:06.571515+0000","last_clean":"2024-04-18T23:12:06.571515+0000","last_became_active":"2024-04-18T23:11:39.923911+0000","last_became_peered":"2024-04-18T23:11:39.923911+0000","last_unstale":"2024-04-18T23:12:06.571515+0000","last_undegraded":"2024-04-18T23:12:06.571515+0000","last_fullsized":"2024-04-18T23:12:06.571515+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001107884,"stat_sum":{"num_bytes":8749056,"num_objects":10,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":40,"num_read_kb":12302,"num_write":64,"num_write_kb":14000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5685248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":164,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.741370+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:12:05.741370+0000","last_peered":"2024-04-18T23:12:05.741370+0000","last_clean":"2024-04-18T23:12:05.741370+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:12:05.741370+0000","last_undegraded":"2024-04-18T23:12:05.741370+0000","last_fullsized":"2024-04-18T23:12:05.741370+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"98'117","reported_seq":422,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.594097+0000","last_change":"2024-04-18T23:12:05.654635+0000","last_active":"2024-04-18T23:12:06.594097+0000","last_peered":"2024-04-18T23:12:06.594097+0000","last_clean":"2024-04-18T23:12:06.594097+0000","last_became_active":"2024-04-18T23:10:54.793319+0000","last_became_peered":"2024-04-18T23:10:54.793319+0000","last_unstale":"2024-04-18T23:12:06.594097+0000","last_undegraded":"2024-04-18T23:12:06.594097+0000","last_fullsized":"2024-04-18T23:12:06.594097+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:20.152899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00069787099999999997,"stat_sum":{"num_bytes":13844480,"num_objects":10,"num_object_clones":6,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":1,"num_read":18,"num_read_kb":9,"num_write":40,"num_write_kb":14224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":156,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.570938+0000","last_change":"2024-04-18T23:10:45.600969+0000","last_active":"2024-04-18T23:12:05.570938+0000","last_peered":"2024-04-18T23:12:05.570938+0000","last_clean":"2024-04-18T23:12:05.570938+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:12:05.570938+0000","last_undegraded":"2024-04-18T23:12:05.570938+0000","last_fullsized":"2024-04-18T23:12:05.570938+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.503236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"99'242","reported_seq":543,"reported_epoch":99,"state":"active+clean+snaptrim","last_fresh":"2024-04-18T23:12:06.661158+0000","last_change":"2024-04-18T23:12:05.943268+0000","last_active":"2024-04-18T23:12:06.661158+0000","last_peered":"2024-04-18T23:12:06.661158+0000","last_clean":"2024-04-18T23:12:06.661158+0000","last_became_active":"2024-04-18T23:10:54.693609+0000","last_became_peered":"2024-04-18T23:10:54.693609+0000","last_unstale":"2024-04-18T23:12:06.661158+0000","last_undegraded":"2024-04-18T23:12:06.661158+0000","last_fullsized":"2024-04-18T23:12:06.661158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":242,"log_dups_size":0,"ondisk_log_size":242,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":1,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:13.582185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013388440000000001,"stat_sum":{"num_bytes":19152896,"num_objects":20,"num_object_clones":12,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":1,"num_read":51,"num_read_kb":11706,"num_write":183,"num_write_kb":21216,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"}]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.594097+0000","last_change":"2024-04-18T23:10:45.493259+0000","last_active":"2024-04-18T23:12:06.594097+0000","last_peered":"2024-04-18T23:12:06.594097+0000","last_clean":"2024-04-18T23:12:06.594097+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:12:06.594097+0000","last_undegraded":"2024-04-18T23:12:06.594097+0000","last_fullsized":"2024-04-18T23:12:06.594097+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"87'42","reported_seq":391,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-18T23:12:05.752978+0000","last_change":"2024-04-18T23:12:05.853594+0000","last_active":"2024-04-18T23:12:05.752978+0000","last_peered":"2024-04-18T23:12:05.752978+0000","last_clean":"2024-04-18T23:12:05.752978+0000","last_became_active":"2024-04-18T23:11:39.717443+0000","last_became_peered":"2024-04-18T23:11:39.717443+0000","last_unstale":"2024-04-18T23:12:05.752978+0000","last_undegraded":"2024-04-18T23:12:05.752978+0000","last_fullsized":"2024-04-18T23:12:05.752978+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:13.038125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.100779232,"stat_sum":{"num_bytes":3358720,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":21,"num_read_kb":7544,"num_write":18,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"d","length":"1"},{"start":"f","length":"1"},{"start":"17","length":"1"},{"start":"1e","length":"2"},{"start":"23","length":"1"}]},{"pgid":"2.7","version":"0'0","reported_seq":84,"reported_epoch":100,"state":"active+clean","last_fresh":"2024-04-18T23:12:06.571293+0000","last_change":"2024-04-18T23:11:33.519671+0000","last_active":"2024-04-18T23:12:06.571293+0000","last_peered":"2024-04-18T23:12:06.571293+0000","last_clean":"2024-04-18T23:12:06.571293+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:12:06.571293+0000","last_undegraded":"2024-04-18T23:12:06.571293+0000","last_fullsized":"2024-04-18T23:12:06.571293+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:07.626952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":135921664,"num_objects":129,"num_object_clones":79,"num_object_copies":903,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":5,"num_read":448,"num_read_kb":146633,"num_write":841,"num_write_kb":152928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237445120,"data_stored":362332160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0},"log_size":1431,"ondisk_log_size":1431,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808840,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39508,"kb_used_data":10940,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94332332,"statfs":{"total":96636764160,"available":96596307968,"internally_reserved":0,"allocated":11202560,"data_stored":16809255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280533,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44024,"kb_used_data":12840,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327816,"statfs":{"total":96636764160,"available":96591683584,"internally_reserved":0,"allocated":13148160,"data_stored":17163444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280533,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42440,"kb_used_data":11260,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329400,"statfs":{"total":96636764160,"available":96593305600,"internally_reserved":0,"allocated":11530240,"data_stored":15320788,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,5],"upper_bound":128},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313236,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58156,"kb_used_data":26976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94313684,"statfs":{"total":96636764160,"available":96577212416,"internally_reserved":0,"allocated":27623424,"data_stored":40629428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313236,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":49452,"kb_used_data":18272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94322388,"statfs":{"total":96636764160,"available":96586125312,"internally_reserved":0,"allocated":18710528,"data_stored":29672628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313237,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37016,"kb_used_data":9996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334824,"statfs":{"total":96636764160,"available":96598859776,"internally_reserved":0,"allocated":10235904,"data_stored":13665460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313237,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54124,"kb_used_data":22944,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94317716,"statfs":{"total":96636764160,"available":96581341184,"internally_reserved":0,"allocated":23494656,"data_stored":34436276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313237,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37620,"kb_used_data":10596,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94334220,"statfs":{"total":96636764160,"available":96598241280,"internally_reserved":0,"allocated":10850304,"data_stored":14701748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313235,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52212,"kb_used_data":21032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94319628,"statfs":{"total":96636764160,"available":96583299072,"internally_reserved":0,"allocated":21536768,"data_stored":31421620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313235,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":45120,"kb_used_data":13940,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326720,"statfs":{"total":96636764160,"available":96590561280,"internally_reserved":0,"allocated":14274560,"data_stored":20796596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313236,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":42456,"kb_used_data":11276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94329384,"statfs":{"total":96636764160,"available":96593289216,"internally_reserved":0,"allocated":11546624,"data_stored":18537531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280533,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":43780,"kb_used_data":12604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94328060,"statfs":{"total":96636764160,"available":96591933440,"internally_reserved":0,"allocated":12906496,"data_stored":17960539,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280533,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":54620,"kb_used_data":23440,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94317220,"statfs":{"total":96636764160,"available":96580833280,"internally_reserved":0,"allocated":24002560,"data_stored":36902068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280534,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47460,"kb_used_data":16272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324380,"statfs":{"total":96636764160,"available":96588165120,"internally_reserved":0,"allocated":16662528,"data_stored":24136999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,0,0,0,0,2],"upper_bound":512},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280533,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46360,"kb_used_data":15168,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325480,"statfs":{"total":96636764160,"available":96589291520,"internally_reserved":0,"allocated":15532032,"data_stored":23416103,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280533,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41460,"kb_used_data":10272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94330380,"statfs":{"total":96636764160,"available":96594309120,"internally_reserved":0,"allocated":10518528,"data_stored":14089511,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":26161152,"data_stored":39284736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":19386368,"data_stored":31440896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":10928128,"data_stored":16146432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8773632,"data_stored":12369920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":22032384,"data_stored":33091584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9388032,"data_stored":13844480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":22540288,"data_stored":36044800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15183872,"data_stored":23269376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20787200,"data_stored":32374784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16502784,"data_stored":25726976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":10350592,"data_stored":14532608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":12812288,"data_stored":19218432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":11591680,"data_stored":19181568,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":9535488,"data_stored":13496320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":11685888,"data_stored":16306176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":9785344,"data_stored":16003072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0}]}} 2024-04-18T23:12:11.636 INFO:tasks.ceph.ceph_manager.ceph:pg 3.9 in trimming, state: active+clean+snaptrim 2024-04-18T23:12:11.636 INFO:tasks.ceph.ceph_manager.ceph:pg 3.7 in trimming, state: active+clean+snaptrim 2024-04-18T23:12:11.636 INFO:tasks.ceph.ceph_manager.ceph:unique_pool_0 still trimming, waiting 2024-04-18T23:12:11.713 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:11.710+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:11.759 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:12:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:12:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:12:11.759 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:176 3.8s0 3:1090b2d1:::smithi14230322-15:head [copy-from ver 6] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.218238888000002, 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.53110666500000003, 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 176 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.760 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539636+0000", 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539638+0000", 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3829999999999999e-06 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.761 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539651+0000", 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2549e-05 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539675+0000", 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4413000000000002e-05 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539689+0000", 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4161999999999999e-05 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.762 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539831+0000", 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014226500000000001 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op must wait for map", 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539865+0000", 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3185000000000002e-05 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644415+0000", 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10455017599999999 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.763 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644498+0000", 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3568999999999999e-05 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.865827+0000", 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.22132859799999999 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.868846+0000", 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030189309999999999 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.764 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070576+0000", 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20172967 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:01.070742+0000", 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016676399999999999 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:177 3.8s0 3:1090b2d1:::smithi14230322-15:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.218156256, 2024-04-18T23:12:11.765 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32703084799999999, 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 177 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.766 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539718+0000", 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539719+0000", 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.37e-07 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539723+0000", 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1620000000000001e-06 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.767 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539733+0000", 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4630000000000007e-06 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.539739+0000", 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7880000000000001e-06 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540029+0000", 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029001100000000001 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.768 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.540050+0000", 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1033e-05 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644678+0000", 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10462763999999999 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644714+0000", 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6202999999999999e-05 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.769 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866546+0000", 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.22183165399999999 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866650+0000", 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000104465 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.866749+0000", 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8791999999999999e-05 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.770 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:207 3.8s0 3:1c19a06b:::smithi14230322-28:head [copy-from ver 4] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.196540088999996, 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.40709379200000001, 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 207 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.771 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561334+0000", 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561336+0000", 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.638e-06 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.772 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561354+0000", 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7413000000000001e-05 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561372+0000", 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8003e-05 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561383+0000", 2024-04-18T23:12:11.773 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1425e-05 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561495+0000", 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000111661 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561506+0000", 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.113e-05 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.774 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644748+0000", 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.083242308000000001 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644807+0000", 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8909999999999997e-05 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.854271+0000", 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.209463753 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.775 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.856288+0000", 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0020173909999999999 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.968263+0000", 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.111975256 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.968428+0000", 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016490400000000001 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.776 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:208 3.8s0 3:1c19a06b:::smithi14230322-28:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 71.196466999999998, 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29446849200000003, 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 208 2024-04-18T23:12:11.777 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.778 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561408+0000", 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4899999999999995e-07 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561411+0000", 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9179999999999998e-06 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561421+0000", 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4599999999999992e-06 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.779 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561427+0000", 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4370000000000004e-06 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561582+0000", 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000154652 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.561592+0000", 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.004e-05 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644937+0000", 2024-04-18T23:12:11.780 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.083345591999999996 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for blocked object", 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.644974+0000", 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6835000000000002e-05 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855704+0000", 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.21072980599999999 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855787+0000", 2024-04-18T23:12:11.781 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3442000000000005e-05 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:00.855876+0000", 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8660999999999997e-05 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:331 3.8s0 3:1c19a06b:::smithi14230322-28:head [stat,delete] snapc 8=[8,7,4] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 60.328987236000003, 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205262474, 2024-04-18T23:12:11.782 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 331 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.783 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428887+0000", 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428890+0000", 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4830000000000002e-06 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428901+0000", 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0852999999999999e-05 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428933+0000", 2024-04-18T23:12:11.784 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2234999999999999e-05 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.428956+0000", 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3436000000000001e-05 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.429116+0000", 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015961600000000001 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.429402+0000", 2024-04-18T23:12:11.785 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028557099999999998 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.431719+0000", 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002317117 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.434261+0000", 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025422980000000001 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.633920+0000", 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.199659052 2024-04-18T23:12:11.786 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:11.634150+0000", 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00022981299999999999 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 47/31 ECSubWriteReply(tid=23, last_complete=47'16, committed=1, applied=1))", 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 59.026799660000002, 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30085146299999999, 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.787 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731075+0000", 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731076+0000", 2024-04-18T23:12:11.788 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.6599999999999994e-07 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731085+0000", 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6670000000000002e-06 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731102+0000", 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7368999999999999e-05 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:12.731113+0000", 2024-04-18T23:12:11.789 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1552999999999999e-05 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031861+0000", 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30074756000000002 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:13.031926+0000", 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5347999999999998e-05 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.790 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:556 3.8s0 3:1c19a06b:::smithi14230322-28:head [append 0~0] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.572903+0000", 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.184971732999998, 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5168125699999999, 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 556 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.791 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572903+0000", 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572903+0000", 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572906+0000", 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6970000000000002e-06 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.792 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572916+0000", 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0839000000000001e-05 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572947+0000", 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0433000000000002e-05 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572962+0000", 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5645999999999999e-05 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573159+0000", 2024-04-18T23:12:11.793 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000196949 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573165+0000", 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7470000000000002e-06 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.877595+0000", 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3044293819999999 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.878153+0000", 2024-04-18T23:12:11.794 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00055816700000000004 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.879453+0000", 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013006300000000001 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886421+0000", 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0069672680000000004 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089573+0000", 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20315222899999999 2024-04-18T23:12:11.795 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089715+0000", 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000142583 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:557 3.8s0 3:1c19a06b:::smithi14230322-28:head [rollback 14] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.572993+0000", 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.184882027999997, 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7177589360000001, 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.796 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 557 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572993+0000", 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572993+0000", 2024-04-18T23:12:11.797 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572993+0000", 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9299999999999996e-07 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.572999+0000", 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.9100000000000002e-06 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573010+0000", 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1129e-05 2024-04-18T23:12:11.798 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573018+0000", 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.9740000000000002e-06 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573204+0000", 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018570899999999999 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573205+0000", 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.057e-06 2024-04-18T23:12:11.799 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.883595+0000", 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3103893069999999 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.883773+0000", 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017896500000000001 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.885247+0000", 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014739029999999999 2024-04-18T23:12:11.800 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089301+0000", 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20405347400000001 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.290623+0000", 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201322048 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.290751+0000", 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000128567 2024-04-18T23:12:11.801 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:558 3.8s0 3:1c19a06b:::smithi14230322-28:head [append 0~0] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.573062+0000", 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.184812182999998, 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.818743402, 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.802 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 558 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573062+0000", 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573062+0000", 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573063+0000", 2024-04-18T23:12:11.803 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2600000000000001e-07 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573067+0000", 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3900000000000002e-06 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573084+0000", 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.77e-05 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.573093+0000", 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.7430000000000008e-06 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.804 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.638122+0000", 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.065028498000000004 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for peered", 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.638126+0000", 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5530000000000003e-06 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.986937+0000", 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3488105429999999 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.805 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.987167+0000", 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023003800000000001 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.988303+0000", 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011358970000000001 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089734+0000", 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101431382 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.806 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.391689+0000", 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30195531799999997 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.391806+0000", 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000116514 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 71/70 ECSubWriteReply(tid=32, last_complete=71'22, committed=1, applied=1))", 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.886489+0000", 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.871385994000001, 2024-04-18T23:12:11.807 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202604601, 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886489+0000", 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886489+0000", 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.808 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886490+0000", 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7770000000000001e-06 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886501+0000", 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0353000000000001e-05 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886529+0000", 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8343e-05 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.809 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.886549+0000", 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9726000000000001e-05 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089054+0000", 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20250496000000001 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.089093+0000", 2024-04-18T23:12:11.810 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9441999999999999e-05 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:634 3.8s0 3:1090b2d1:::smithi14230322-15:head [create,setxattr UrPUWbaU]cNfinVtoVUNsYX\\ahn\\oSdRW\\N[RcfO (807) in=40b,setxattr ZQoeVnpeaooblfNZklrg\\hmphZ^pTVh`eRXheade (1225) in=40b,setxattr kcjR][ah`rcmsWjUVlUeaofblgmpleUjjs^]ai\\t (7928) in=40b,setxattr `U^gRRs]UWRVngUaNeN^`p`aVl\\cn_a\\S`iTpOVo (1156) in=40b,setxattr aOYbW^`[gQfVcTQrq[NYeqh\\Q_bjiaWOefngkiNR (1331) in=40b,setxattr igNjm (735) in=5b,setxattr aQYs^c[hafOm^\\g[^R\\^QOd^eiS\\ghimncRcb_pi (15856) in=40b,setxattr dZUpqXmQYdS]obbkNRYdSsdUpenRkjiSSfUmqRTd (9612) in=40b,setxattr lX^flfV^[d^WmlW_UT^kV`s^ahfcNQsT`de`^spi (1612) in=40b,setxattr Ua`UXimY^dWVoQaji\\cZpP^er^hlbYQZs`USR`oh (10033) in=40b,setxattr XhOccedqUYhrPkk[^dVtangb_r^]alTl`ogaOPag (840) in=40b,setxattr YqbXbb\\oPeeR]rqcNaSsS`lkYoffXPcpaqhrb^te (418) in=40b,setxattr jdOeY]PQnfd[iZ^ph\\ijd[WRRnQ^phTpRROYeXss (1611) in=40b,setxattr ie\\ahhoaskiWrasYNh\\mUkqRso\\Olj\\c^P_\\bPp\\ (15577) in=40b,setxattr Ptb\\gnjjhTnd`OQag[dXTYa]t_id[PRpjt_ZfVY] (1541) in=40b,setxattr sSR]a[NPVaUQhPkla[ZVX`lTdim`fgjgeRtm`tt^ (841) in=40b,setxattr _cf[gnNmUkknhenPpY^\\lefOR_n\\mcb`iqOOPUiW (11296) in=40b,setxattr d[TVh[TTre^P`mnfob^WP[dckRkQZafjR[PaW^Zn (8912) in=40b,setxattr X[gWOjQtntoVfbZV[UObOVPoj`qib[[Zt]trgVjh (1542) in=40b,setxattr _qa\\VSbX\\\\b`tO_lSRnsf`PVNfmPkWrf[P\\ZTej[ (453) in=40b] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.865771+0000", 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 27.892103179999999, 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30787136700000001, 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.811 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 634 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865771+0000", 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865771+0000", 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.812 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865774+0000", 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9000000000000002e-06 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865868+0000", 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3826999999999995e-05 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865902+0000", 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3562000000000002e-05 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.813 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.865918+0000", 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6189e-05 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.966141+0000", 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10022311 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.966365+0000", 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000223556 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.814 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.968971+0000", 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0026065469999999999 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.071518+0000", 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102546892 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.173465+0000", 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10194739799999999 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.173643+0000", 2024-04-18T23:12:11.815 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000177386 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 82/31 ECSubWrite(tid=312, reqid=osd.4.0:308, at_version=0'0, trim_to=0'0, roll_forward_to=81'162))", 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.083084+0000", 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.674790659999999, 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30191269900000001, 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.816 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083084+0000", 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083084+0000", 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083085+0000", 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4160000000000001e-06 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083116+0000", 2024-04-18T23:12:11.817 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0664999999999999e-05 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.083146+0000", 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9689e-05 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183319+0000", 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100173805 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183491+0000", 2024-04-18T23:12:11.818 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000171757 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183521+0000", 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9581999999999999e-05 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.384935+0000", 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201414449 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.384997+0000", 2024-04-18T23:12:11.819 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1335999999999997e-05 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s5 82/31 ECSubWrite(tid=314, reqid=osd.4.0:314, at_version=82'166, trim_to=0'0, roll_forward_to=81'162))", 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.183484+0000", 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.574390360999999, 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20244278099999999, 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.820 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183484+0000", 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183484+0000", 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183485+0000", 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.7999999999999995e-07 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.821 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183516+0000", 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1684999999999999e-05 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183529+0000", 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2872e-05 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.183539+0000", 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8500000000000006e-06 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.822 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.384381+0000", 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.200841872 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.384401+0000", 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9639e-05 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.385858+0000", 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001457243 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.823 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.385927+0000", 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9040000000000003e-05 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:690 3.8s0 3:1090b2d1:::smithi14230322-15:head [append 0~704512] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.807395+0000", 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.950479604000002, 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.51943692100000005, 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.824 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 690 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.807395+0000", 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.807395+0000", 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.825 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.807398+0000", 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.253e-06 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.808935+0000", 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001536412 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.808969+0000", 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4051000000000003e-05 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.826 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.808985+0000", 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6186000000000001e-05 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.909266+0000", 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100281412 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.909664+0000", 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00039748600000000001 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.827 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.913002+0000", 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0033379619999999999 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.024906+0000", 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.111904329 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.326692+0000", 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30178585099999999 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.326832+0000", 2024-04-18T23:12:11.828 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013997900000000001 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:691 3.8s0 3:1090b2d1:::smithi14230322-15:head [append 0~655360] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.814196+0000", 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.943678883, 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.61312221899999997, 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.829 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 691 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.814196+0000", 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.814196+0000", 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.830 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.814197+0000", 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4529999999999999e-06 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.815127+0000", 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00092968399999999996 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.815148+0000", 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1671999999999999e-05 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.831 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915240+0000", 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100091456 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915549+0000", 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000308547 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915731+0000", 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018246000000000001 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.917602+0000", 2024-04-18T23:12:11.832 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001871277 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.125440+0000", 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.207837408 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427190+0000", 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30175016300000002 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427318+0000", 2024-04-18T23:12:11.833 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012809900000000001 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:692 3.8s0 3:1090b2d1:::smithi14230322-15:head [append 0~507904] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.915310+0000", 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.842564579000001, 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.51235423099999999, 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.834 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 692 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915310+0000", 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915310+0000", 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.835 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.915312+0000", 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0870000000000001e-06 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916047+0000", 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00073506099999999996 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916075+0000", 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7657999999999998e-05 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.836 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916088+0000", 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3217000000000001e-05 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.919103+0000", 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0030152550000000001 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.919395+0000", 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00029170500000000001 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.837 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.921531+0000", 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021356930000000001 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.125444+0000", 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20391351699999999 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427541+0000", 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30209633800000002 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.838 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427664+0000", 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001237 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:693 3.8s0 3:1090b2d1:::smithi14230322-15:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.916109+0000", 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.841765464000002, 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.54009619900000005, 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.839 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 693 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916109+0000", 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916109+0000", 2024-04-18T23:12:11.840 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916110+0000", 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0699999999999999e-06 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916115+0000", 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.242e-06 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916126+0000", 2024-04-18T23:12:11.841 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0414000000000001e-05 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916136+0000", 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0349e-05 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.023380+0000", 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10724339400000001 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.842 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.023571+0000", 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00019098699999999999 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.024464+0000", 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00089320100000000004 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.225628+0000", 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201164015 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:11.843 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.455934+0000", 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.23030629799999999 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.456205+0000", 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000271229 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:694 3.8s0 3:1090b2d1:::smithi14230322-15:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.916170+0000", 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.841704944, 2024-04-18T23:12:11.844 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.74335174800000003, 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 694 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916170+0000", 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.845 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916170+0000", 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916171+0000", 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.33e-07 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916176+0000", 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8799999999999996e-06 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.916187+0000", 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0949e-05 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.016296+0000", 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100108797 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.125250+0000", 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10895395200000001 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.125424+0000", 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000174091 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.457293+0000", 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.33186873900000002 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.457553+0000", 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00026042900000000001 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.659521+0000", 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20196797799999999 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 83/70 ECSubWriteReply(tid=60, last_complete=83'53, committed=1, applied=1))", 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.116515+0000", 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.641360041999999, 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20973929499999999, 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.849 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116515+0000", 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116515+0000", 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116516+0000", 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.469e-06 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116524+0000", 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.4840000000000004e-06 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116551+0000", 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6721999999999999e-05 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.116589+0000", 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8268e-05 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.326219+0000", 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20962908399999999 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.326254+0000", 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5268000000000002e-05 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWriteReply(3.8s0 83/70 ECSubWriteReply(tid=60, last_complete=83'53, committed=1, applied=1))", 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.172192+0000", 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.585682636000001, 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.25482231500000002, 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172192+0000", 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:11.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172192+0000", 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172193+0000", 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.55e-06 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172202+0000", 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.636e-06 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:11.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172228+0000", 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5879000000000001e-05 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.172241+0000", 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2903e-05 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.426978+0000", 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.25473670199999998 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.427014+0000", 2024-04-18T23:12:11.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6644999999999999e-05 2024-04-18T23:12:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:11.856 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:11.856 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:11.858 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:12:11.858 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:11.855+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:11.958 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:11.954+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:11.984 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:12:11.984 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:12:11.984 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:11.998 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:12:12.058 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:12.059+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:12.122 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:12.122 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:12:12.122 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:12:12.122 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:12:12.122 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:12.136 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:12:12.159 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:12.159+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:12.257 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:12:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:12:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:12:12.257 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 57/31 ECSubWrite(tid=167, reqid=osd.4.0:167, at_version=57'77, trim_to=0'0, roll_forward_to=56'73))", 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:22.631249+0000", 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.624515506999998, 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20201448899999999, 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631249+0000", 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.258 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631249+0000", 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631250+0000", 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0559999999999999e-06 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631287+0000", 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7069e-05 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.259 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.631308+0000", 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0723000000000001e-05 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.731401+0000", 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100093711 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.831671+0000", 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100269671 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.260 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.831687+0000", 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6283999999999999e-05 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.833165+0000", 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0014771879999999999 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.833263+0000", 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.8787000000000001e-05 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.261 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 57/31 ECSubWrite(tid=170, reqid=osd.4.0:167, at_version=0'0, trim_to=0'0, roll_forward_to=57'77))", 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:22.834221+0000", 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 49.421543225000001, 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30088188700000001, 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.262 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834221+0000", 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834221+0000", 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834223+0000", 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.516e-06 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.263 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834241+0000", 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8712999999999999e-05 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.834271+0000", 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9963000000000001e-05 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.934412+0000", 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10014097499999999 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.264 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.934586+0000", 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000173535 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:22.934601+0000", 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5574999999999999e-05 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:23.135038+0000", 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20043703500000001 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.265 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:23.135103+0000", 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4574999999999998e-05 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 61/31 ECSubWrite(tid=190, reqid=osd.4.0:188, at_version=0'0, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.458871+0000", 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.796893724, 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20139204499999999, 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.266 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458871+0000", 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458871+0000", 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458872+0000", 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5740000000000001e-06 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.267 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458892+0000", 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0060000000000001e-05 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458921+0000", 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8714999999999999e-05 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.458936+0000", 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5296000000000001e-05 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.559175+0000", 2024-04-18T23:12:12.268 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100238271 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.559198+0000", 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3133e-05 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.660126+0000", 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100928437 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.660263+0000", 2024-04-18T23:12:12.269 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013655899999999999 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 62/31 ECSubWrite(tid=191, reqid=client.4568.0:474, at_version=62'89, trim_to=0'0, roll_forward_to=61'87))", 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:27.579357+0000", 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 44.676407091000002, 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205588628, 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.270 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.271 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579357+0000", 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579357+0000", 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.273 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.579358+0000", 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7300000000000002e-07 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.580046+0000", 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00068822000000000002 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.580081+0000", 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4882000000000002e-05 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.274 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680226+0000", 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100145083 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680407+0000", 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000181068 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.680437+0000", 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9983000000000001e-05 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.275 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.784862+0000", 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.104425411 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:27.784946+0000", 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.3608000000000003e-05 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 70/31 ECSubWrite(tid=60, reqid=client.4568.0:554, at_version=70'46, trim_to=0'0, roll_forward_to=70'44))", 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:36.879929+0000", 2024-04-18T23:12:12.276 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 35.375835608999999, 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20180873899999999, 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.879929+0000", 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.879929+0000", 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.277 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.879931+0000", 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.956e-06 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.879974+0000", 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.3755999999999999e-05 2024-04-18T23:12:12.278 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.880012+0000", 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.7276e-05 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:36.980154+0000", 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100141984 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.080426+0000", 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100272718 2024-04-18T23:12:12.279 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.080443+0000", 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6382999999999999e-05 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.081661+0000", 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001218418 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.081737+0000", 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6248000000000006e-05 2024-04-18T23:12:12.280 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpRead(3.6s6 70/68 ECSubRead(tid=36, to_read={3:635103ff:::smithi14230322-31:head=0,262144,0}, subchunks={3:635103ff:::smithi14230322-31:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.390943+0000", 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.864820831000003, 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30072780199999999, 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.281 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390943+0000", 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390943+0000", 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390945+0000", 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.88e-06 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.282 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390963+0000", 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7180000000000002e-05 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.390997+0000", 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4616000000000002e-05 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.491149+0000", 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100151937 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.283 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.691524+0000", 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20037495599999999 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.691671+0000", 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000147233 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.as3 71/31 ECSubWrite(tid=19, reqid=osd.3.0:19, at_version=71'8, trim_to=0'0, roll_forward_to=69'6))", 2024-04-18T23:12:12.284 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:37.577755+0000", 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 34.678008871999999, 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20409171300000001, 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577755+0000", 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577755+0000", 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.285 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577756+0000", 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6399999999999998e-07 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577795+0000", 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.9125000000000002e-05 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.577818+0000", 2024-04-18T23:12:12.286 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3368999999999999e-05 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.677913+0000", 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10009472799999999 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.778121+0000", 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10020746799999999 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.778137+0000", 2024-04-18T23:12:12.287 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6852999999999999e-05 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.781737+0000", 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0035999230000000001 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:37.781847+0000", 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000109883 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.288 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 72/31 ECSubWrite(tid=66, reqid=osd.2.0:66, at_version=72'52, trim_to=0'0, roll_forward_to=71'48))", 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:38.620535+0000", 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.635229672999998, 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20162907699999999, 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620535+0000", 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.289 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620535+0000", 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620535+0000", 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2600000000000001e-07 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620554+0000", 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8617000000000001e-05 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.290 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.620566+0000", 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1931e-05 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.720651+0000", 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100085153 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.820810+0000", 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100159237 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.291 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.820822+0000", 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1331999999999999e-05 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.822128+0000", 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001305816 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.822164+0000", 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6164999999999997e-05 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.292 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 72/31 ECSubWrite(tid=214, reqid=osd.4.0:214, at_version=72'104, trim_to=0'0, roll_forward_to=66'102))", 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:38.716533+0000", 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.539230928000002, 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202635437, 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716533+0000", 2024-04-18T23:12:12.293 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716533+0000", 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716534+0000", 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8099999999999999e-07 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716557+0000", 2024-04-18T23:12:12.294 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3773000000000001e-05 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716578+0000", 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0633999999999999e-05 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716589+0000", 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.065e-05 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.295 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716738+0000", 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014939799999999999 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.716756+0000", 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7802999999999999e-05 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.919099+0000", 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20234323300000001 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.296 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.919169+0000", 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.9665000000000005e-05 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 72/31 ECSubWrite(tid=216, reqid=osd.4.0:216, at_version=72'106, trim_to=0'0, roll_forward_to=66'102))", 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:38.717905+0000", 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.537859097999998, 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.301742538, 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.297 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.717905+0000", 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.717905+0000", 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.717907+0000", 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.57e-06 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.298 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.718037+0000", 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00012998299999999999 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.718069+0000", 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.1890000000000001e-05 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.718095+0000", 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6072e-05 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.299 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.918419+0000", 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20032456900000001 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.918440+0000", 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0632000000000002e-05 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.019609+0000", 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101169125 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.300 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.019648+0000", 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.8696999999999999e-05 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.es1 72/31 ECSubWrite(tid=67, reqid=osd.2.0:66, at_version=0'0, trim_to=0'0, roll_forward_to=72'52))", 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:38.824136+0000", 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 33.431628697000001, 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20173307800000001, 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.301 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824136+0000", 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824136+0000", 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824137+0000", 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4799999999999997e-07 2024-04-18T23:12:12.302 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824148+0000", 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1686e-05 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824163+0000", 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4545000000000001e-05 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:38.824175+0000", 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2678e-05 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.303 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.024455+0000", 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20027978799999999 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.024472+0000", 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6782999999999998e-05 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.025834+0000", 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0013615999999999999 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.304 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:39.025869+0000", 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.5049999999999998e-05 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 77/31 ECSubWrite(tid=246, reqid=client.4568.0:606, at_version=77'84, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.901108+0000", 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.354656168999998, 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20364768699999999, 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.305 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901108+0000", 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901108+0000", 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901110+0000", 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6700000000000001e-06 2024-04-18T23:12:12.306 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901608+0000", 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049785200000000002 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901633+0000", 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.5055999999999999e-05 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901651+0000", 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8428e-05 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.307 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.001855+0000", 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100203588 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.001877+0000", 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1971999999999999e-05 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.104674+0000", 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102796919 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.308 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.104756+0000", 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2201999999999997e-05 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 77/31 ECSubWrite(tid=247, reqid=client.4568.0:607, at_version=77'85, trim_to=0'0, roll_forward_to=73'82))", 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.901685+0000", 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.354079235, 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20315154799999999, 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.309 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901685+0000", 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901685+0000", 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901686+0000", 2024-04-18T23:12:12.310 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8700000000000004e-07 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901908+0000", 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000221843 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.901926+0000", 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7932e-05 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.002023+0000", 2024-04-18T23:12:12.311 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100097214 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.102305+0000", 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10028247699999999 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.102326+0000", 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0879000000000001e-05 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.104773+0000", 2024-04-18T23:12:12.312 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0024464159999999999 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.104837+0000", 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3899999999999995e-05 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 82/31 ECSubWrite(tid=311, reqid=osd.4.0:311, at_version=82'124, trim_to=0'0, roll_forward_to=81'120))", 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.955944+0000", 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 23.299820094000001, 2024-04-18T23:12:12.313 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202757677, 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.955944+0000", 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.955944+0000", 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.314 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.955946+0000", 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.793e-06 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.955988+0000", 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1681999999999998e-05 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.956017+0000", 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9077999999999998e-05 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.315 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.056165+0000", 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100147999 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.156449+0000", 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10028411499999999 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.156481+0000", 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2094999999999997e-05 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.316 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.158639+0000", 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021584590000000002 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.158702+0000", 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.2456e-05 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.bs6 83/31 ECSubWrite(tid=318, reqid=client.4568.0:695, at_version=83'126, trim_to=0'0, roll_forward_to=82'124))", 2024-04-18T23:12:12.317 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.830934+0000", 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.424830666999998, 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20389481700000001, 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.830934+0000", 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.830934+0000", 2024-04-18T23:12:12.318 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.830936+0000", 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.12e-06 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.831373+0000", 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00043691699999999999 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.831407+0000", 2024-04-18T23:12:12.319 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4771000000000003e-05 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.931510+0000", 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10010235200000001 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.031835+0000", 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10032532 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.031872+0000", 2024-04-18T23:12:12.320 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6514999999999999e-05 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.034733+0000", 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00286141 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.034828+0000", 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.5412000000000001e-05 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.321 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.6s6 83/72 ECSubWrite(tid=53, reqid=client.4568.0:724, at_version=83'40, trim_to=0'0, roll_forward_to=81'36))", 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.991435+0000", 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.264329377999999, 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201353955, 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991435+0000", 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.322 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991435+0000", 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991436+0000", 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3e-07 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991452+0000", 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6852000000000001e-05 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.323 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991464+0000", 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1362000000000001e-05 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.991473+0000", 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.3370000000000006e-06 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.191528+0000", 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20005491 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.324 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.191557+0000", 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9108999999999999e-05 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.192743+0000", 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011859939999999999 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.192789+0000", 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.5760999999999998e-05 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.325 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 84/31 ECSubWrite(tid=337, reqid=client.4568.0:754, at_version=84'177, trim_to=0'0, roll_forward_to=83'173))", 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.964705+0000", 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 21.291059309000001, 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20257407599999999, 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964705+0000", 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.326 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964705+0000", 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964707+0000", 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.8929999999999999e-06 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964903+0000", 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000195873 2024-04-18T23:12:12.327 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.964934+0000", 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0972000000000001e-05 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.065082+0000", 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100148376 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.065242+0000", 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015953199999999999 2024-04-18T23:12:12.328 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.065270+0000", 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.7915e-05 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.167201+0000", 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10193097800000001 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:51.167279+0000", 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.8536999999999997e-05 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.329 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 87/31 ECSubWrite(tid=380, reqid=osd.4.0:376, at_version=0'0, trim_to=0'0, roll_forward_to=87'198))", 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:54.371773+0000", 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.883991095999999, 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20143276099999999, 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.330 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371773+0000", 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371773+0000", 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371775+0000", 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0200000000000001e-06 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371797+0000", 2024-04-18T23:12:12.331 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2146000000000001e-05 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371827+0000", 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0071999999999999e-05 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371844+0000", 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6849000000000001e-05 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.332 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.472133+0000", 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100288628 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.472150+0000", 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7241999999999999e-05 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573153+0000", 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101003211 2024-04-18T23:12:12.333 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573206+0000", 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2593000000000003e-05 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 87/31 ECSubWrite(tid=381, reqid=osd.4.0:381, at_version=87'200, trim_to=0'0, roll_forward_to=87'198))", 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:54.371887+0000", 2024-04-18T23:12:12.334 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.883877291000001, 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202086396, 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371887+0000", 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371887+0000", 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.335 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371888+0000", 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.9699999999999996e-07 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371922+0000", 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4378000000000002e-05 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371939+0000", 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7173e-05 2024-04-18T23:12:12.336 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371950+0000", 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0572e-05 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.572712+0000", 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.200762725 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.572722+0000", 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0012e-05 2024-04-18T23:12:12.337 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573932+0000", 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012091770000000001 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573973+0000", 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.1862e-05 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.338 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "MOSDECSubOpWrite(3.7s2 87/31 ECSubWrite(tid=382, reqid=osd.4.0:382, at_version=87'202, trim_to=0'0, roll_forward_to=87'198))", 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:54.371970+0000", 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 17.883794075000001, 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.202436489, 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "reached pg", 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371970+0000", 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.339 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371970+0000", 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.371971+0000", 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 4.6699999999999999e-07 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.372005+0000", 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4375000000000002e-05 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.340 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.372017+0000", 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1429e-05 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.372026+0000", 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2669999999999995e-06 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573285+0000", 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20125895999999999 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.341 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.573299+0000", 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4156e-05 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.574348+0000", 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001049358 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:54.574407+0000", 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.8477000000000003e-05 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.342 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.343 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.343 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.343 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.343 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:12.344 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:12:12.345 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:12:12.342+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:12:12.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:12:12.445 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:12.442+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:12.469 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:12:12.469 INFO:teuthology.orchestra.run.smithi120.stdout: "num_ops": 0 2024-04-18T23:12:12.470 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:12.484 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:12:12.545 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:12.542+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:12.581 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:12:12.637 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:12:12.638 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:12:12.638 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:12:12.638 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-04-18T23:12:12.638 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:12:12.639 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:12:12.646 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:12.642+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:12.646 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:12:12.646 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:12:12.652 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:12.652 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:12:12.652 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:12:12.652 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:12:12.652 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:12.663 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:12:12.663 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:12:12.667 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:12:12.684 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:12:12.684 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:12:12.705 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:12:12.705 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:12:12.726 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:12:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:12:12.744 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:12:12.744 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:12:12.746 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:12.747+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:12.764 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:12:12.764 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:12:12.785 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:12:12.785 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:12:12.807 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:12:12.808 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:12:12.826 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:12:12.826 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:12:12.845 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 85.250861728999993, 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62069844100000005, 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.846 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 114 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.847 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594565+0000", 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594570+0000", 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.186e-06 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594582+0000", 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1763000000000001e-05 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.848 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594590+0000", 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5290000000000003e-06 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010572+0000", 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.415982252 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010727+0000", 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015558400000000001 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.849 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010971+0000", 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000243339 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015929+0000", 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049583639999999998 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215108+0000", 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19917934400000001 2024-04-18T23:12:12.850 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215199+0000", 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0556000000000005e-05 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215263+0000", 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3874000000000005e-05 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.851 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 85.250825402000004, 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62089894099999998, 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:12:12.852 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 115 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:12:12.853 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594746+0000", 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014446700000000001 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594761+0000", 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4792e-05 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.854 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594769+0000", 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6160000000000002e-06 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013742+0000", 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.418972438 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013974+0000", 2024-04-18T23:12:12.855 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023265699999999999 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.014328+0000", 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035345 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018069+0000", 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037411079999999999 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215346+0000", 2024-04-18T23:12:12.856 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19727757900000001 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215435+0000", 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8245999999999995e-05 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215500+0000", 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4968000000000005e-05 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.857 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 85.250642150999994, 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.72133309899999998, 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 116 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.858 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594785+0000", 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:12:12.859 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.860 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594790+0000", 2024-04-18T23:12:12.860 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4229999999999996e-06 2024-04-18T23:12:12.860 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.860 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.860 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594803+0000", 2024-04-18T23:12:12.861 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2196e-05 2024-04-18T23:12:12.861 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.861 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.861 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.861 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594810+0000", 2024-04-18T23:12:12.862 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6190000000000002e-06 2024-04-18T23:12:12.862 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.862 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.862 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.862 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016062+0000", 2024-04-18T23:12:12.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42125235 2024-04-18T23:12:12.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.863 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.863 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.863 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016215+0000", 2024-04-18T23:12:12.863 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000152051 2024-04-18T23:12:12.863 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.864 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:12:12.864 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016420+0000", 2024-04-18T23:12:12.864 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000205916 2024-04-18T23:12:12.864 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.864 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:12:12.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018082+0000", 2024-04-18T23:12:12.865 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001661718 2024-04-18T23:12:12.865 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.865 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.865 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:12:12.865 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.315968+0000", 2024-04-18T23:12:12.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29788616600000001 2024-04-18T23:12:12.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.866 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.866 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.866 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316051+0000", 2024-04-18T23:12:12.866 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2897000000000003e-05 2024-04-18T23:12:12.866 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.867 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.867 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.867 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316117+0000", 2024-04-18T23:12:12.867 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5866999999999998e-05 2024-04-18T23:12:12.867 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.867 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.867 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.868 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.868 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.868 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:12:12.868 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:12:12.868 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 85.250604518000003, 2024-04-18T23:12:12.868 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.721588585, 2024-04-18T23:12:12.868 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.868 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.869 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.869 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.869 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:12:12.869 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:12:12.869 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 117 2024-04-18T23:12:12.869 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.869 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.869 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:12:12.870 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.870 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.870 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.870 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.870 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:12:12.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.871 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.871 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.871 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.871 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:12:12.871 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:12:12.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.872 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.872 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.872 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594829+0000", 2024-04-18T23:12:12.872 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7830000000000001e-06 2024-04-18T23:12:12.872 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.873 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.873 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594841+0000", 2024-04-18T23:12:12.873 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1505000000000001e-05 2024-04-18T23:12:12.873 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.873 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.874 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.694940+0000", 2024-04-18T23:12:12.874 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100099585 2024-04-18T23:12:12.874 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.874 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.874 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016967+0000", 2024-04-18T23:12:12.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32202692799999999 2024-04-18T23:12:12.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.875 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.875 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017224+0000", 2024-04-18T23:12:12.875 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000256548 2024-04-18T23:12:12.875 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.875 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:12:12.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017583+0000", 2024-04-18T23:12:12.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000359689 2024-04-18T23:12:12.876 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.876 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.876 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:12:12.876 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.020139+0000", 2024-04-18T23:12:12.876 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025556649999999999 2024-04-18T23:12:12.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:12:12.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316256+0000", 2024-04-18T23:12:12.877 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29611688800000002 2024-04-18T23:12:12.877 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.877 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.877 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.877 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316345+0000", 2024-04-18T23:12:12.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9285999999999999e-05 2024-04-18T23:12:12.878 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.878 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.878 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.878 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316410+0000", 2024-04-18T23:12:12.878 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5058e-05 2024-04-18T23:12:12.879 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.879 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.879 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.879 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.879 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.879 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:26 3.2s0 3:4250b374:::smithi14230322-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:12:12.879 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:12:12.880 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.321218492, 2024-04-18T23:12:12.880 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60852922899999995, 2024-04-18T23:12:12.880 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.880 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.880 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.880 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.880 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.881 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.881 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 26 2024-04-18T23:12:12.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.881 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.881 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.881 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.881 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:12:12.881 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.881 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524210+0000", 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.882 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524220+0000", 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0056e-05 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524242+0000", 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2212e-05 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525734+0000", 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001491783 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.883 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528099+0000", 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023654380000000001 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528318+0000", 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000219065 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.529513+0000", 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011952530000000001 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.884 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636414+0000", 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106900906 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132457+0000", 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49604301899999997 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132737+0000", 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027950099999999998 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.885 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:31 3.2s0 3:42c34e84:::smithi14230322-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.319104068000001, 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60793644899999999, 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.886 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 31 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.887 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.888 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:12:12.888 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.888 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.888 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.888 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526324+0000", 2024-04-18T23:12:12.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9400000000000001e-06 2024-04-18T23:12:12.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.890 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.890 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.890 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526836+0000", 2024-04-18T23:12:12.890 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051192200000000005 2024-04-18T23:12:12.890 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526869+0000", 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2978000000000002e-05 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526890+0000", 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481999999999999e-05 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.891 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531167+0000", 2024-04-18T23:12:12.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042762490000000002 2024-04-18T23:12:12.892 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.892 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.892 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.892 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531508+0000", 2024-04-18T23:12:12.892 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034109400000000001 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.533620+0000", 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021122279999999998 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636418+0000", 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102798509 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.893 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134075+0000", 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.497656453 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134259+0000", 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000183594 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:32 3.2s0 3:42c34e84:::smithi14230322-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 76.318501158000004, 2024-04-18T23:12:12.894 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60874687999999999, 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 32 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.895 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526927+0000", 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0389999999999999e-06 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526936+0000", 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2599999999999994e-06 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.896 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0302e-05 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526973+0000", 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6359e-05 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634473+0000", 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.107499583 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.897 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634639+0000", 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016687200000000001 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.635580+0000", 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094034999999999995 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636830+0000", 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012506220000000001 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.898 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135509+0000", 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49867835399999999 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135672+0000", 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000163139 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:618 3.2s0 3:42c34e84:::smithi14230322-11:head [read 0~1835008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.223182241, 2024-04-18T23:12:12.899 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.63639373399999999, 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 618 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.900 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622245+0000", 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622253+0000", 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4889999999999998e-06 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.901 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622276+0000", 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2938999999999998e-05 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722418+0000", 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100141811 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822632+0000", 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100214391 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.902 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:12.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822645+0000", 2024-04-18T23:12:12.903 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2863e-05 2024-04-18T23:12:12.903 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.903 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.903 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.903 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.841571+0000", 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018926235 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.841706+0000", 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000134528 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.258638+0000", 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41693209799999997 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.904 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:619 3.2s0 3:42c34e84:::smithi14230322-11:head [read 0~1835008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.122966681000001, 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.94106379600000001, 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 619 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.905 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722462+0000", 2024-04-18T23:12:12.906 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.12e-06 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722475+0000", 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3475999999999999e-05 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722500+0000", 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4817999999999999e-05 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722517+0000", 2024-04-18T23:12:12.907 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7452000000000002e-05 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822716+0000", 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100198302 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822729+0000", 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3036e-05 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.908 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.842152+0000", 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019423458000000001 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.842286+0000", 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013423100000000001 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.663523+0000", 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.82123690299999996 2024-04-18T23:12:12.909 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:628 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~589824] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 29.006311289999999, 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.82267251500000005, 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.910 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 628 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.911 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839118+0000", 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6759999999999999e-06 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840018+0000", 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090027100000000001 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840051+0000", 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3426999999999997e-05 2024-04-18T23:12:12.912 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840069+0000", 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7869000000000001e-05 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.949596+0000", 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10952640800000001 2024-04-18T23:12:12.913 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.949748+0000", 2024-04-18T23:12:12.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001526 2024-04-18T23:12:12.914 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.914 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.914 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.914 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.951307+0000", 2024-04-18T23:12:12.914 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001558633 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.052856+0000", 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101548762 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.661580+0000", 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.608724088 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.661787+0000", 2024-04-18T23:12:12.915 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020778100000000001 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:629 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~540672] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.998681788999999, 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91730556200000002, 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.916 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 629 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:12:12.917 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.918 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.918 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.918 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846746+0000", 2024-04-18T23:12:12.918 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5799999999999999e-06 2024-04-18T23:12:12.918 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.919 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847461+0000", 2024-04-18T23:12:12.919 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071526900000000004 2024-04-18T23:12:12.919 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.919 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.919 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847485+0000", 2024-04-18T23:12:12.920 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3509000000000001e-05 2024-04-18T23:12:12.920 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.920 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.920 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.920 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847497+0000", 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2675e-05 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.047393+0000", 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19989558700000001 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.047586+0000", 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000192568 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.921 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.049081+0000", 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001495225 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.052942+0000", 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038612070000000002 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.763898+0000", 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.71095591800000002 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.922 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764050+0000", 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015202400000000001 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:630 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~622592] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.989819804, 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.908871761, 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.923 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.924 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.924 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.924 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 630 2024-04-18T23:12:12.924 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.924 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.924 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.924 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855609+0000", 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.17e-06 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.925 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856431+0000", 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082242899999999996 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856463+0000", 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2320999999999998e-05 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856479+0000", 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5396e-05 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.926 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.050092+0000", 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19361344699999999 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.050246+0000", 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000153497 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.051908+0000", 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001662562 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.927 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.154762+0000", 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102854035 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764322+0000", 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60956000700000001 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764478+0000", 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000155897 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.928 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:631 3.2s0 3:4b92d38e:::smithi14230322-19:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.988920136000001, 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90835886899999996, 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 631 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.929 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856507+0000", 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1000000000000001e-06 2024-04-18T23:12:12.930 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856512+0000", 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2290000000000002e-06 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856534+0000", 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1356e-05 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856543+0000", 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8060000000000001e-06 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.931 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.053052+0000", 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19650978299999999 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.053209+0000", 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015716999999999999 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054047+0000", 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000837925 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.932 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.154833+0000", 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100785849 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764707+0000", 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60987347400000003 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764865+0000", 2024-04-18T23:12:12.933 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015817699999999999 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:632 3.2s0 3:4b92d38e:::smithi14230322-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 28.988870633000001, 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.212644217, 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.934 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.935 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:12.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.935 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.935 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 632 2024-04-18T23:12:12.935 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.935 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.935 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.935 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0699999999999996e-07 2024-04-18T23:12:12.936 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856560+0000", 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4230000000000001e-06 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856573+0000", 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3009e-05 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.937 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856581+0000", 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0669999999999992e-06 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054652+0000", 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198071467 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054999+0000", 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034629799999999998 2024-04-18T23:12:12.938 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.766030+0000", 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.71103122399999996 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.766186+0000", 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015654300000000001 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:45.069200+0000", 2024-04-18T23:12:12.939 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30301327900000002 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:670 3.2s0 3:4250b374:::smithi14230322-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.664265+0000", 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 24.181161483, 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62706280400000003, 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.940 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 670 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664265+0000", 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664265+0000", 2024-04-18T23:12:12.941 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664267+0000", 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2239999999999998e-06 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664279+0000", 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1919000000000001e-05 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664310+0000", 2024-04-18T23:12:12.942 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0855000000000002e-05 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664331+0000", 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1614999999999999e-05 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664485+0000", 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015335899999999999 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664494+0000", 2024-04-18T23:12:12.943 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4660000000000005e-06 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.088025+0000", 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.423531251 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.088204+0000", 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017829 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.944 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.291327+0000", 2024-04-18T23:12:12.945 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20312382500000001 2024-04-18T23:12:12.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.945 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.945 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.945 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.945 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.945 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:741 3.2s0 3:42341748:::smithi14230322-4:head [append 0~425984] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.968667+0000", 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.876759203999999, 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91216083299999995, 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 741 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.946 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.968667+0000", 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.968667+0000", 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.968670+0000", 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8559999999999998e-06 2024-04-18T23:12:12.947 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.969441+0000", 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077078300000000004 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.969477+0000", 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6038000000000003e-05 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069573+0000", 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10009589100000001 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.948 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070763+0000", 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011900960000000001 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071013+0000", 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025074299999999999 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.073185+0000", 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002171849 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.949 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.270509+0000", 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19732339800000001 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.880686+0000", 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.61017750400000004 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.880828+0000", 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014167499999999999 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.950 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:742 3.2s0 3:42341748:::smithi14230322-4:head [append 0~606208] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.069661+0000", 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.775764971000001, 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.81171484500000002, 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.951 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 742 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069661+0000", 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069661+0000", 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069664+0000", 2024-04-18T23:12:12.952 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8930000000000001e-06 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070530+0000", 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086640600000000001 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070560+0000", 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9538000000000001e-05 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070581+0000", 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0483e-05 2024-04-18T23:12:12.953 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.074800+0000", 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042195740000000002 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.075058+0000", 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025823000000000003 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.954 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.077007+0000", 2024-04-18T23:12:12.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019490169999999999 2024-04-18T23:12:12.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.955 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.955 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278274+0000", 2024-04-18T23:12:12.955 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201266476 2024-04-18T23:12:12.955 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.955 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881251+0000", 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60297683199999996 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881376+0000", 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125396 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.956 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:743 3.2s0 3:42341748:::smithi14230322-4:head [append 0~540672] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.070618+0000", 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.774807997, 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.81111058300000005, 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 743 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.957 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070618+0000", 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070618+0000", 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070620+0000", 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9640000000000002e-06 2024-04-18T23:12:12.958 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071703+0000", 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001083044 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071733+0000", 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9699000000000001e-05 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071752+0000", 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9275000000000001e-05 2024-04-18T23:12:12.959 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.270584+0000", 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19883204800000001 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.270766+0000", 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018226000000000001 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.960 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.272279+0000", 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015124030000000001 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.279437+0000", 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071585529999999998 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881609+0000", 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60217155700000002 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.961 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881729+0000", 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011977999999999999 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:744 3.2s0 3:42341748:::smithi14230322-4:head [append 0~409600] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.071789+0000", 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.773637582999999, 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91037796400000004, 2024-04-18T23:12:12.962 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 744 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071789+0000", 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.963 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071789+0000", 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071791+0000", 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1459999999999999e-06 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072442+0000", 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065123700000000004 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.964 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072461+0000", 2024-04-18T23:12:12.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9485999999999999e-05 2024-04-18T23:12:12.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.965 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.965 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072479+0000", 2024-04-18T23:12:12.965 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7892000000000001e-05 2024-04-18T23:12:12.965 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.965 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.966 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.273674+0000", 2024-04-18T23:12:12.966 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20119500400000001 2024-04-18T23:12:12.966 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.966 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.966 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.273959+0000", 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028440799999999999 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.277315+0000", 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003356412 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.281415+0000", 2024-04-18T23:12:12.967 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041003360000000004 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982028+0000", 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.70061242700000004 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982167+0000", 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013861599999999999 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.968 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:745 3.2s0 3:42341748:::smithi14230322-4:head [setxattr _header (56) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.072506+0000", 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 22.772919999999999, 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91007206600000001, 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 745 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:12.969 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072506+0000", 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072506+0000", 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072508+0000", 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4470000000000001e-06 2024-04-18T23:12:12.970 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072523+0000", 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5756e-05 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072543+0000", 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9712e-05 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072559+0000", 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5758999999999999e-05 2024-04-18T23:12:12.971 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278314+0000", 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205755145 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278426+0000", 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011250499999999999 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278921+0000", 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049428099999999995 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.972 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.281726+0000", 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028056639999999998 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982417+0000", 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.70069052600000004 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982578+0000", 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016127099999999999 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.973 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.974 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:12.974 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:12.974 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:12.975 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:12:12.975 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:12:12.976 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:12:12.976 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:12:12.976 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:12:12.977 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:12:12.978 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:12:12.979 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:12.975+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:13.076 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:13.074+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:13.110 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:13.110 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:13.125 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:12:13.177 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:13.176+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:13.242 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:13.242 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:13.256 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:12:13.277 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:13.275+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:13.378 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:13.374+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:13.425 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:13.425 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:13.439 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:12:13.480 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:13.478+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:13.564 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:13.564 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:13.578 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:12:13.581 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:12:13.578+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:12:13.682 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:13.679+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:13.701 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:13.701 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:13.715 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:12:13.782 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:13.778+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:13.836 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:13.836 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:13.850 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:12:13.883 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:13.882+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:13.966 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:13.966 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:13.980 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:12:13.984 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:13.982+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:14.084 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:14.082+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:14.098 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:14.098 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:14.114 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:12:14.186 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:14.186+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:14.234 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:14.234 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:14.248 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:12:14.287 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:14.287+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:14.374 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:14.388 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:12:14.388 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:14.387+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:14.489 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:14.486+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:14.507 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:14.508 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:14.522 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:12:14.590 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:14.588+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:14.626 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15] out_osds: [] dead_osds: [] live_osds: [0, 4, 8, 12, 1, 5, 9, 13, 2, 6, 10, 14, 3, 7, 11, 15] 2024-04-18T23:12:14.626 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-18T23:12:14.640 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-18T23:12:14.646 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:14.647 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:14.661 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:12:14.706 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:12:14.707+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:12:14.792 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:12:14.792+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:12:14.795 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:14.795 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:14.809 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:12:14.893 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:14.890+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:14.977 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:14.978 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:14.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:12:14.992 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:12:14.993 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:14.990+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:15.094 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:12:15.090+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:12:15.117 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:15.117 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:15.123 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:12:15.123 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:12:15.131 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:12:15.144 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:12:15.144 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:12:15.165 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:12:15.165 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:12:15.184 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:12:15.184 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:12:15.195 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:15.194+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:15.200 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:12:15.201 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:12:15.221 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:12:15.221 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:12:15.242 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:12:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:12:15.254 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:15.254 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:15.263 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:12:15.263 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:12:15.269 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:12:15.282 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:12:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:12:15.304 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:12:15.304 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:12:15.314 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:15.315+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:15.325 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:12:15.326 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:12:15.344 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:12:15.345 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:12:15.362 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:12:15.362 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:12:15.383 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:12:15.384 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:12:15.392 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:15.392 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:12:15.392 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:12:15.392 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:12:15.392 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:15.397 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:15.396+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:15.405 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:12:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:12:15.407 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:12:15.425 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:12:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:12:15.498 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:15.499+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:202 3.ds0 3:ba4438ef:::smithi14230322-48:head [copy-from ver 2] snapc 2=[2] ondisk+write+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 74.925371894999998, 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.726850518, 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.573 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 202 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.574 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647076+0000", 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647077+0000", 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.497e-06 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647086+0000", 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5490000000000006e-06 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.575 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647105+0000", 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8933e-05 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.647120+0000", 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4942e-05 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752033+0000", 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10491311 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.576 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.752334+0000", 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00030070900000000002 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.072768+0000", 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.32043411399999999 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.075547+0000", 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0027797260000000002 2024-04-18T23:12:15.577 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.373671+0000", 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.29812367699999998 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.373926+0000", 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025526100000000001 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.578 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:323 3.ds0 3:b8bdcfdb:::smithi14230322-44:head [delete] snapc 8=[8,7,4] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e46)", 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 64.962586790000003, 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81471389100000002, 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 323 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.579 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609861+0000", 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609863+0000", 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6360000000000001e-06 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.580 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609869+0000", 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.4699999999999999e-06 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609890+0000", 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0496000000000002e-05 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.609905+0000", 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5539000000000001e-05 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.581 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.614202+0000", 2024-04-18T23:12:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0042970489999999998 2024-04-18T23:12:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:10.614219+0000", 2024-04-18T23:12:15.582 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7373000000000001e-05 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119137+0000", 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50491712899999996 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.119293+0000", 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015688099999999999 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.583 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.120709+0000", 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014158669999999999 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.122707+0000", 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0019980140000000002 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424382+0000", 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30167486399999999 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.584 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:11.424575+0000", 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000192573 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:390 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~475136] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.675363+0000", 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 58.897084614000001, 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.80904401299999995, 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.585 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 390 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675363+0000", 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675363+0000", 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.586 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675365+0000", 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1569999999999998e-06 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675936+0000", 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00057051499999999995 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675968+0000", 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1779000000000002e-05 2024-04-18T23:12:15.587 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.675983+0000", 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5313000000000001e-05 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.873642+0000", 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.197659001 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.873655+0000", 2024-04-18T23:12:15.588 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3073e-05 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.877399+0000", 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0037437019999999998 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.877612+0000", 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021331900000000001 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.589 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.879219+0000", 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001607266 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.881917+0000", 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0026975240000000002 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484280+0000", 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60236371499999997 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.590 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484407+0000", 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000126649 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:391 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [append 0~294912] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.677038+0000", 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 58.895409825999998, 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.80775349699999999, 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.591 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 391 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677038+0000", 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677038+0000", 2024-04-18T23:12:15.592 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677040+0000", 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0059999999999999e-06 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677406+0000", 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00036561500000000002 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.677440+0000", 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4972999999999999e-05 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.593 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777555+0000", 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100114034 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.880210+0000", 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10265596 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.880389+0000", 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017841299999999999 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.594 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.881177+0000", 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00078838699999999999 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.183072+0000", 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30189508599999998 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484661+0000", 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30158817300000001 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.595 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.484791+0000", 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013085000000000001 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:393 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.777705+0000", 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 58.794742978999999, 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0113445000000001, 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.596 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 393 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777705+0000", 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777705+0000", 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.597 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777706+0000", 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.626e-06 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777716+0000", 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0146000000000001e-05 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777742+0000", 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5236e-05 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.598 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777762+0000", 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9879999999999999e-05 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.982379+0000", 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20461718700000001 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.982705+0000", 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00032662899999999999 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.599 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.486129+0000", 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50342405499999998 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.486297+0000", 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016788799999999999 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.789049+0000", 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30275185300000002 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.600 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:703 3.ds0 3:b210783d:::smithi14230322-30:head [append 0~737280] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.852388+0000", 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 25.720059812999999, 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71389459600000005, 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 703 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.601 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.852388+0000", 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.852388+0000", 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.852391+0000", 2024-04-18T23:12:15.602 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8399999999999999e-06 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853230+0000", 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00083946900000000002 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853263+0000", 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2502999999999997e-05 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853278+0000", 2024-04-18T23:12:15.603 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5619000000000001e-05 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853408+0000", 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000129617 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.853578+0000", 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000170366 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.855600+0000", 2024-04-18T23:12:15.604 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0020215070000000001 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.858840+0000", 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00324041 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566135+0000", 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.70729440899999996 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566282+0000", 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000147856 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.605 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:707 3.ds0 3:b210783d:::smithi14230322-30:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.961197+0000", 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 25.611251192000001, 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81019067, 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 707 2024-04-18T23:12:15.606 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961197+0000", 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961197+0000", 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961198+0000", 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5200000000000001e-06 2024-04-18T23:12:15.607 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961208+0000", 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.022e-05 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961228+0000", 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0078999999999999e-05 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.961244+0000", 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5566e-05 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.608 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.164394+0000", 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20314963 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.164576+0000", 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018204500000000001 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.568877+0000", 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.404301311 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.609 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.569042+0000", 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016466199999999999 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.771387+0000", 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20234563699999999 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:907 3.ds0 3:ba4438ef:::smithi14230322-48:head [append 0~425984] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.610 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.576464+0000", 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.995983487, 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.996576193, 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 907 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.576464+0000", 2024-04-18T23:12:15.611 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.576464+0000", 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.576466+0000", 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.08e-06 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.612 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.576696+0000", 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022981600000000001 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.576729+0000", 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2515000000000003e-05 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.576750+0000", 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1243999999999999e-05 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.613 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.672930+0000", 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.096180159000000001 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.672940+0000", 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.8859999999999999e-06 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.680654+0000", 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0077136269999999998 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.614 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.680968+0000", 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031478099999999998 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.682480+0000", 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001511713 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.685751+0000", 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003271262 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.615 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.572896+0000", 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.88714427299999998 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.573040+0000", 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014483700000000001 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:908 3.ds0 3:ba4438ef:::smithi14230322-48:head [append 0~638976] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.583097+0000", 2024-04-18T23:12:15.616 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.989351041000001, 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0903842640000001, 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 908 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.583097+0000", 2024-04-18T23:12:15.617 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.583097+0000", 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.583099+0000", 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0760000000000001e-06 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.618 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.583464+0000", 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00036575000000000003 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.583497+0000", 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2183e-05 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.583512+0000", 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5705999999999998e-05 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.619 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.675555+0000", 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.092042524000000001 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.675563+0000", 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1319999999999994e-06 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.683352+0000", 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0077886420000000001 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.683519+0000", 2024-04-18T23:12:15.620 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016723000000000001 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.684769+0000", 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012499150000000001 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.685960+0000", 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011913220000000001 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.621 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.673346+0000", 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.98738588000000005 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.673481+0000", 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013490400000000001 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:909 3.ds0 3:ba4438ef:::smithi14230322-48:head [append 0~491520] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.588155+0000", 2024-04-18T23:12:15.622 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.984292458000001, 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0857071309999999, 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 909 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.588155+0000", 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.588155+0000", 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.588157+0000", 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.21e-06 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.588730+0000", 2024-04-18T23:12:15.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00057253999999999996 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.588763+0000", 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2555999999999998e-05 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.688849+0000", 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100086437 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.688974+0000", 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012484799999999999 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689272+0000", 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029819699999999999 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.691592+0000", 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0023201760000000002 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.801399+0000", 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.109806315 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.673732+0000", 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.87233343399999996 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.673862+0000", 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013041800000000001 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.627 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:910 3.ds0 3:ba4438ef:::smithi14230322-48:head [setxattr _header (57) in=7b] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.688886+0000", 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.883562066, 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0851153309999999, 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 910 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.628 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.688886+0000", 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.688886+0000", 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.688889+0000", 2024-04-18T23:12:15.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8899999999999999e-06 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.688929+0000", 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.0379000000000001e-05 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.688954+0000", 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4813000000000001e-05 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.688971+0000", 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.717e-05 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.693005+0000", 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040336649999999996 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.693179+0000", 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017425000000000001 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.694148+0000", 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00096910900000000001 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.801482+0000", 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.107333886 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.773841+0000", 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97235956899999998 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.774001+0000", 2024-04-18T23:12:15.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0001596 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:911 3.ds0 3:ba4438ef:::smithi14230322-48:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.689004+0000", 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.883443758, 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.191737662, 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.633 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 911 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689004+0000", 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689004+0000", 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689006+0000", 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.61e-06 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689013+0000", 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.4399999999999999e-06 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689032+0000", 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9432999999999998e-05 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689048+0000", 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5126000000000001e-05 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.694813+0000", 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0057652279999999998 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.695162+0000", 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00034924700000000001 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.774142+0000", 2024-04-18T23:12:15.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0789797299999999 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.774313+0000", 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000171347 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.880742+0000", 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10642850099999999 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:914 3.ds0 3:bb40a4d8:::smithi14230322-41:head [append 0~671744] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.689104+0000", 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.88334401, 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0862912170000001, 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 914 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.638 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689104+0000", 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689104+0000", 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.689105+0000", 2024-04-18T23:12:15.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5069999999999999e-06 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.690061+0000", 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00095567600000000003 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.690097+0000", 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6439999999999997e-05 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.690118+0000", 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0758999999999999e-05 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.795373+0000", 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105254668 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.795551+0000", 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017852100000000001 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.797153+0000", 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001601608 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.801485+0000", 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0043319769999999999 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.775232+0000", 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97374705100000003 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.775395+0000", 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016301 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:915 3.ds0 3:bb40a4d8:::smithi14230322-41:head [append 0~573440] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.690153+0000", 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.882294585, 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0859046109999999, 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.643 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 915 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.690153+0000", 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.690153+0000", 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.690155+0000", 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2960000000000001e-06 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.690995+0000", 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00083987199999999997 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.691024+0000", 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8683e-05 2024-04-18T23:12:15.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.691045+0000", 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0676999999999999e-05 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.798375+0000", 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.107330098 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.798536+0000", 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016159 2024-04-18T23:12:15.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.800197+0000", 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0016604740000000001 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.802425+0000", 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0022280239999999999 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.775807+0000", 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.97338225700000003 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.776058+0000", 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025064 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:916 3.ds0 3:bb40a4d8:::smithi14230322-41:head [append 0~753664] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.691071+0000", 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.881377005000001, 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0853678470000001, 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.648 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 916 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.691071+0000", 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.691071+0000", 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.691072+0000", 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.497e-06 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692142+0000", 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001070117 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692172+0000", 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9427000000000001e-05 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692191+0000", 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9335999999999999e-05 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.802643+0000", 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.11045213 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.802901+0000", 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000257712 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.805003+0000", 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002101782 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.811411+0000", 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0064087040000000003 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.776318+0000", 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.96490678399999996 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.776439+0000", 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000120358 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:917 3.ds0 3:bb40a4d8:::smithi14230322-41:head [append 0~229376] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.692223+0000", 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.880225202, 2024-04-18T23:12:15.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.084591267, 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 917 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692223+0000", 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.654 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692223+0000", 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692224+0000", 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.637e-06 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692579+0000", 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00035529000000000002 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692604+0000", 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.4973e-05 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692633+0000", 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8316000000000001e-05 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.806333+0000", 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.113699829 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.806508+0000", 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017553299999999999 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.807777+0000", 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012690240000000001 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.812450+0000", 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0046730319999999997 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.776686+0000", 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.96423588900000001 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.776814+0000", 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000127744 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:918 3.ds0 3:bb40a4d8:::smithi14230322-41:head [setxattr _header (57) in=7b] snapc 24=[24,23,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.658 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.692666+0000", 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.879781749999999, 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.084499549, 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 918 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692666+0000", 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692666+0000", 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692668+0000", 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.73e-06 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692676+0000", 2024-04-18T23:12:15.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.3599999999999996e-06 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692698+0000", 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2393000000000001e-05 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692717+0000", 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8810000000000001e-05 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.809071+0000", 2024-04-18T23:12:15.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.116353911 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.809293+0000", 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022158599999999999 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.810335+0000", 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001041915 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.812460+0000", 2024-04-18T23:12:15.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0021248560000000001 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.777019+0000", 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.96455951100000004 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.777166+0000", 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000146477 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:919 3.ds0 3:bb40a4d8:::smithi14230322-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.692738+0000", 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.879709391, 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.392924088, 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 919 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.664 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692738+0000", 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692738+0000", 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692740+0000", 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5030000000000001e-06 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692746+0000", 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2700000000000001e-06 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692769+0000", 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2413e-05 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.692784+0000", 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5153e-05 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.811044+0000", 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.118260012 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.811385+0000", 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00034124699999999998 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.779428+0000", 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.96804323599999997 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.779649+0000", 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022132299999999999 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.085662+0000", 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30601293099999999 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(osd.13.22:2 3.ds0 3:b7d0d2d3:::smithi14230322-10:head [assert-version v104,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e98)", 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:04.699341+0000", 2024-04-18T23:12:15.668 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 10.873106328, 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.77421099100000002, 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.699341+0000", 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.699341+0000", 2024-04-18T23:12:15.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.699344+0000", 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5670000000000002e-06 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.699355+0000", 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1029999999999999e-05 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.699388+0000", 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2815999999999999e-05 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.699403+0000", 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5653000000000001e-05 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.811471+0000", 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.112067817 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:04.811674+0000", 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000202437 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:05.473552+0000", 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.66187867099999997 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:950 3.ds0 3:bb40a4d8:::smithi14230322-41:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.598948+0000", 2024-04-18T23:12:15.672 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 8.9734993050000007, 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.73661649900000004, 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 950 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.598948+0000", 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.598948+0000", 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.598950+0000", 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4959999999999999e-06 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.598956+0000", 2024-04-18T23:12:15.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.314e-06 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.598975+0000", 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8729999999999999e-05 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.598990+0000", 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4809e-05 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.698986+0000", 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099996465000000007 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.699274+0000", 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028825900000000002 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.085884+0000", 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.386609272 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.086008+0000", 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012430199999999999 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.335565+0000", 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.249556852 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:15.677 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:15.678 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.0 to 1 (10000) 2024-04-18T23:12:15.679 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is set 2024-04-18T23:12:15.680 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:12:15.680 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-18T23:12:15.682 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:15.678+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:15.779 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:15.778+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:15.852 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:15.852 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:12:15.852 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:12:15.852 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:15.866 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:12:15.880 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:12:15.874+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:12:15.980 INFO:tasks.ceph.osd.12.smithi073.stderr:2024-04-18T23:12:15.978+0000 7ff3a6962640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 12 (PID: 25178) UID: 0 2024-04-18T23:12:15.987 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:15.987 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:12:15.987 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:12:15.987 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:12:15.987 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:16.002 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:12:16.082 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:16.080+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:16.127 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:16.127 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:12:16.127 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:192 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 75.565005021000005, 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30501581700000002, 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 192 2024-04-18T23:12:16.128 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560438+0000", 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560439+0000", 2024-04-18T23:12:16.129 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6029999999999999e-06 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560450+0000", 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0835999999999999e-05 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560478+0000", 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7619000000000001e-05 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.130 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560489+0000", 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0998999999999999e-05 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560680+0000", 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00019082800000000001 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560688+0000", 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.2260000000000005e-06 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.131 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662105+0000", 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10141672 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662220+0000", 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000115872 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.865454+0000", 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20323311399999999 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.132 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:193 3.cs0 3:35606aaf:::smithi14230322-20:2 [read 0~327680,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 75.564896275999999, 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30581751899999998, 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 193 2024-04-18T23:12:16.133 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560547+0000", 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560548+0000", 2024-04-18T23:12:16.134 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6299999999999993e-07 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560554+0000", 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2230000000000004e-06 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560566+0000", 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2622999999999999e-05 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560575+0000", 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.1589999999999993e-06 2024-04-18T23:12:16.135 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563654+0000", 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0030794709999999999 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.563665+0000", 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0708e-05 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.662682+0000", 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.099017665000000005 2024-04-18T23:12:16.136 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.663181+0000", 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049841999999999996 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.866364+0000", 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20318328699999999 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.137 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:210 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 75.564798667999995, 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.41278097800000002, 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 210 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.138 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560644+0000", 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560645+0000", 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1829999999999999e-06 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.139 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560652+0000", 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.3500000000000002e-06 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560669+0000", 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6833e-05 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560682+0000", 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3585e-05 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.140 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560856+0000", 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017400200000000001 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560874+0000", 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8386e-05 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767285+0000", 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20641079700000001 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.141 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767423+0000", 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013740399999999999 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.973425+0000", 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20600243800000001 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:211 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~720896,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:16.142 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 75.564710371999993, 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51049911299999995, 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 211 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.143 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560732+0000", 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560734+0000", 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-06 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560741+0000", 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.1559999999999998e-06 2024-04-18T23:12:16.144 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560758+0000", 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7382999999999999e-05 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560775+0000", 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7051999999999999e-05 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560940+0000", 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164659 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.145 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.560949+0000", 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7630000000000006e-06 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.767943+0000", 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.206994276 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:00.768107+0000", 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000164358 2024-04-18T23:12:16.146 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:01.071232+0000", 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30312427600000003 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:359 3.cs0 3:35606aaf:::smithi14230322-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 62.929194422000002, 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.36957785599999998, 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.147 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 359 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196248+0000", 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.148 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196249+0000", 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5700000000000002e-07 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196253+0000", 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8979999999999998e-06 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196271+0000", 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8270999999999998e-05 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.149 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196280+0000", 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.85e-06 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196650+0000", 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00037011599999999998 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.196698+0000", 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.8060999999999999e-05 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.150 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565575+0000", 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.36887705999999998 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565695+0000", 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000119663 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:13.565826+0000", 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013118 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.151 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:384 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~442368] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 59.460901118000002, 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51544569399999995, 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 384 2024-04-18T23:12:16.152 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.153 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664542+0000", 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.664544+0000", 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6240000000000002e-06 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.154 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665037+0000", 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00049266399999999997 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665070+0000", 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2963000000000003e-05 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665089+0000", 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8672e-05 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665233+0000", 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014389900000000001 2024-04-18T23:12:16.155 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.665245+0000", 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2446000000000001e-05 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769742+0000", 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.104497225 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.770038+0000", 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029553099999999997 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.156 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.772939+0000", 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0029013009999999998 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781282+0000", 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0083427859999999996 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.179880+0000", 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.398598591 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.157 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.179987+0000", 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00010699199999999999 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:385 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~720896] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 59.456827492999999, 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61243552499999998, 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.158 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 385 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668615+0000", 2024-04-18T23:12:16.159 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.668618+0000", 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5129999999999999e-06 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669576+0000", 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000957726 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.160 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669605+0000", 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9566000000000001e-05 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669622+0000", 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6409000000000002e-05 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.775317+0000", 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.105695331 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.161 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.775501+0000", 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018403000000000001 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.777261+0000", 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001759652 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781284+0000", 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0040233049999999996 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.162 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.280844+0000", 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49956001900000002 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281051+0000", 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020697399999999999 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:386 3.fs0 3:f47c3018:::smithi14230322-43:head [append 0~49152] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:12:16.163 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 59.455790423000003, 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61210873300000002, 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 386 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.164 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669652+0000", 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669654+0000", 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.297e-06 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669730+0000", 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6474e-05 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.165 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.669748+0000", 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7598999999999999e-05 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769883+0000", 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100135671 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.778786+0000", 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0089029639999999993 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.166 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.779085+0000", 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029830699999999999 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.780258+0000", 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011728070000000001 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783446+0000", 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031888179999999999 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.167 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281593+0000", 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49814646800000001 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.281761+0000", 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016832800000000001 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:387 3.fs0 3:f47c3018:::smithi14230322-43:head [setxattr _header (57) in=7b] snapc c=[c,b,a,9,7,4] ondisk+write+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 59.355519866999998, 2024-04-18T23:12:16.168 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51243170900000001, 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 387 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.169 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769923+0000", 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769925+0000", 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7799999999999999e-06 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769934+0000", 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.6460000000000005e-06 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.170 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769956+0000", 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1216000000000002e-05 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769967+0000", 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-05 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781530+0000", 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.01156212 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.171 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.781688+0000", 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015813200000000001 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.782502+0000", 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00081454100000000001 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783689+0000", 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001186552 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.172 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.282098+0000", 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49840956600000003 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.282355+0000", 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025625600000000001 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:388 3.fs0 3:f47c3018:::smithi14230322-43:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e51)", 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 59.355462439, 2024-04-18T23:12:16.173 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.81817408400000002, 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 388 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.174 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769980+0000", 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769981+0000", 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.6400000000000001e-07 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769985+0000", 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7689999999999998e-06 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.175 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.769996+0000", 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.075e-05 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.770003+0000", 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5769999999999996e-06 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783045+0000", 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.013041308999999999 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:16.176 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:16.783303+0000", 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00025812900000000001 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.283544+0000", 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50024145900000005 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.283825+0000", 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00028086099999999998 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.177 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.588154+0000", 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30432946599999999 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpReadReply(3.fs0 51/31 ECSubReadReply(tid=28, attrs_read=0))", 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 58.840350972000003, 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30201136499999998, 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:16.178 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285092+0000", 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285093+0000", 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.08e-06 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.179 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285111+0000", 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8272000000000001e-05 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285129+0000", 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7706999999999998e-05 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.285140+0000", 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1472e-05 2024-04-18T23:12:16.180 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587038+0000", 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30189753600000002 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:17.587103+0000", 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5297999999999997e-05 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.181 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWriteReply(3.fs0 61/31 ECSubWriteReply(tid=39, last_complete=61'15, committed=1, applied=1))", 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:26.653658+0000", 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 49.471784958000001, 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50232716700000002, 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653658+0000", 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.182 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653658+0000", 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653660+0000", 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2170000000000001e-06 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653670+0000", 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0183000000000001e-05 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.183 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653689+0000", 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.9026e-05 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:26.653701+0000", 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1922999999999999e-05 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:27.154982+0000", 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50128102200000002 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:27.155985+0000", 2024-04-18T23:12:16.184 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001002796 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:561 3.cs0 3:35606aaf:::smithi14230322-20:head [append 0~442368] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.524261+0000", 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 37.601182299000001, 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.31889435799999999, 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 561 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.185 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524261+0000", 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524261+0000", 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524263+0000", 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8040000000000002e-06 2024-04-18T23:12:16.186 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524799+0000", 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00053593200000000003 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524834+0000", 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4693000000000001e-05 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.524852+0000", 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7612999999999999e-05 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.187 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.525052+0000", 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020047899999999999 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.525066+0000", 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3937000000000001e-05 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.629867+0000", 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10480112 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.630105+0000", 2024-04-18T23:12:16.188 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00023814499999999999 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.631631+0000", 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0015257669999999999 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.641942+0000", 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.010310784 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.842982+0000", 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20104003100000001 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.189 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.843155+0000", 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000173053 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:562 3.cs0 3:35606aaf:::smithi14230322-20:head [append 0~540672] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.531229+0000", 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 37.594213979000003, 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61875124800000003, 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.190 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 562 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531229+0000", 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.191 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531229+0000", 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531231+0000", 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0530000000000001e-06 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.531984+0000", 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00075264199999999998 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.192 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.532012+0000", 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.8932999999999999e-05 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.532029+0000", 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6203000000000001e-05 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.638598+0000", 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.106569199 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.193 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.638913+0000", 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031543000000000001 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.640689+0000", 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001775715 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.742333+0000", 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101644006 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.194 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.149762+0000", 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40742915800000001 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.149980+0000", 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021790899999999999 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:563 3.cs0 3:35606aaf:::smithi14230322-20:head [append 0~622592] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.538968+0000", 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 37.586474744999997, 2024-04-18T23:12:16.195 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.68520978200000004, 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 563 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.538968+0000", 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.196 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.538968+0000", 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.538970+0000", 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0559999999999999e-06 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539699+0000", 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00072919900000000004 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.197 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539729+0000", 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.9899999999999998e-05 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539744+0000", 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4708999999999999e-05 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.943419+0000", 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.403674637 2024-04-18T23:12:16.198 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.943605+0000", 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018626 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.945303+0000", 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001697893 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148746+0000", 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20344289300000001 2024-04-18T23:12:16.199 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.224013+0000", 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.075267071000000005 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.224178+0000", 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016516400000000001 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.200 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:564 3.cs0 3:35606aaf:::smithi14230322-20:head [setxattr _header (57) in=7b] snapc 16=[15,14,13,12,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.539769+0000", 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 37.585673425000003, 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.78455714300000001, 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 564 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.201 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539769+0000", 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539769+0000", 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539771+0000", 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2470000000000001e-06 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.202 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539777+0000", 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.0730000000000001e-06 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539788+0000", 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0924000000000001e-05 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539795+0000", 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.8159999999999994e-06 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.046902+0000", 2024-04-18T23:12:16.203 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50710653999999999 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.047084+0000", 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000181609 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.047936+0000", 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00085221099999999996 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148748+0000", 2024-04-18T23:12:16.204 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100812007 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.324156+0000", 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17540829599999999 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.324327+0000", 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017042000000000001 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:565 3.cs0 3:35606aaf:::smithi14230322-20:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e72)", 2024-04-18T23:12:16.205 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:38.539807+0000", 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 37.585635838999998, 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0887034069999999, 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 565 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.206 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539807+0000", 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539807+0000", 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539808+0000", 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9299999999999997e-07 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.207 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539811+0000", 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5700000000000001e-06 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539822+0000", 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0467e-05 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:38.539829+0000", 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.5870000000000004e-06 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.208 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148486+0000", 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60865698899999998 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.148648+0000", 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016127099999999999 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.326655+0000", 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.17800708000000001 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.209 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.326971+0000", 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00031639400000000001 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.628510+0000", 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30153925599999998 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "MOSDECSubOpWrite(3.7s6 72/31 ECSubWrite(tid=224, reqid=osd.4.0:222, at_version=0'0, trim_to=0'0, roll_forward_to=72'111))", 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:39.511725+0000", 2024-04-18T23:12:16.210 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 36.613718190999997, 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30299252599999998, 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "reached pg", 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511725+0000", 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511725+0000", 2024-04-18T23:12:16.211 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511726+0000", 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1999999999999998e-07 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511748+0000", 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2276000000000002e-05 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.511772+0000", 2024-04-18T23:12:16.212 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.3932999999999999e-05 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.611922+0000", 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100150379 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.712249+0000", 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100326822 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.712286+0000", 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6962999999999998e-05 2024-04-18T23:12:16.213 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.814658+0000", 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.102371988 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:39.814717+0000", 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9244999999999999e-05 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.214 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:943 3.cs0 3:35606aaf:::smithi14230322-20:head [copy-from ver 181] snapc 25=[25,24,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.572755+0000", 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.5526874680000002, 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.56518704099999995, 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 943 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.215 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572755+0000", 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572755+0000", 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572758+0000", 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0060000000000001e-06 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.216 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572773+0000", 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4346e-05 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572808+0000", 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5206999999999997e-05 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572824+0000", 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6449000000000001e-05 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.217 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572916+0000", 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1841000000000004e-05 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572926+0000", 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5680000000000005e-06 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.576305+0000", 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003379506 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.218 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.576611+0000", 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000305858 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.919156+0000", 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.34254441800000002 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.927045+0000", 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0078894800000000008 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.219 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.137819+0000", 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.21077352899999999 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.137942+0000", 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012383300000000001 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:944 3.cs0 3:35606aaf:::smithi14230322-20:head [stat] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.572857+0000", 2024-04-18T23:12:16.220 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.5525859670000006, 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.35365487800000001, 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 944 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572857+0000", 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.221 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572857+0000", 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572858+0000", 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1540000000000001e-06 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572864+0000", 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.1229999999999997e-06 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.222 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572876+0000", 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2262999999999999e-05 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.572885+0000", 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.789e-06 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.573015+0000", 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000129515 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.223 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.573030+0000", 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5696999999999999e-05 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.577046+0000", 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.004015785 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for blocked object", 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.577144+0000", 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7862000000000006e-05 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.224 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.926321+0000", 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.34917678000000002 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.926406+0000", 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.5292999999999998e-05 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.926512+0000", 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000105617 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.225 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.226 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.226 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.226 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:16.227 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:12:16.228 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:16.228+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:16.328 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:16.328+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:16.353 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:16.353 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:12:16.353 INFO:teuthology.orchestra.run.smithi142.stdout: "num_ops": 0 2024-04-18T23:12:16.353 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:16.367 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-18T23:12:16.429 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:16.428+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:16.485 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:16.485 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [], 2024-04-18T23:12:16.485 INFO:teuthology.orchestra.run.smithi142.stdout: "complaint_time": 30, 2024-04-18T23:12:16.485 INFO:teuthology.orchestra.run.smithi142.stdout: "num_blocked_ops": 0 2024-04-18T23:12:16.485 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:16.499 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-18T23:12:16.546 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:16.547+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:16.621 INFO:teuthology.orchestra.run.smithi142.stdout:{ 2024-04-18T23:12:16.621 INFO:teuthology.orchestra.run.smithi142.stdout: "size": 20, 2024-04-18T23:12:16.621 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 600, 2024-04-18T23:12:16.621 INFO:teuthology.orchestra.run.smithi142.stdout: "ops": [ 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:636 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~114688,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:44.611449+0000", 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 32.007838587000002, 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.407104254, 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 636 2024-04-18T23:12:16.622 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611449+0000", 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611449+0000", 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611452+0000", 2024-04-18T23:12:16.623 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4470000000000002e-06 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611468+0000", 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.5716e-05 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611504+0000", 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.5963000000000001e-05 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611522+0000", 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.8275999999999999e-05 2024-04-18T23:12:16.624 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611614+0000", 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.1224999999999995e-05 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "op must wait for map", 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611622+0000", 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.9859999999999994e-06 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.712617+0000", 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100995288 2024-04-18T23:12:16.625 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.712794+0000", 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017712699999999999 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:45.018553+0000", 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.305759226 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.626 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:637 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~114688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:44.611560+0000", 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 32.007727080999999, 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40760569800000002, 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 637 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.627 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611560+0000", 2024-04-18T23:12:16.628 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.628 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.628 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.628 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611560+0000", 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611561+0000", 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6600000000000002e-07 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611564+0000", 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.6909999999999999e-06 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.629 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611569+0000", 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6659999999999999e-06 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611572+0000", 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.174e-06 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611687+0000", 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00011503800000000001 2024-04-18T23:12:16.630 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611693+0000", 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.9270000000000002e-06 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.713563+0000", 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.101869986 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.713746+0000", 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018268700000000001 2024-04-18T23:12:16.631 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:45.019166+0000", 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30542016300000002 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:638 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~114688,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e78)", 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:44.611596+0000", 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 32.007691588999997, 2024-04-18T23:12:16.632 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.45990405099999998, 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 638 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611596+0000", 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.633 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611596+0000", 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611596+0000", 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.6600000000000003e-07 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611598+0000", 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1330000000000002e-06 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.634 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611603+0000", 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.5480000000000003e-06 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.611608+0000", 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.7389999999999999e-06 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.635 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.614722+0000", 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0031144559999999998 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.614728+0000", 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2729999999999999e-06 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.814553+0000", 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.19982428199999999 2024-04-18T23:12:16.636 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:44.814741+0000", 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018874699999999999 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:45.071500+0000", 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.25675860700000003 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.637 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:700 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~0] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.839315+0000", 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.779972941, 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60877221999999998, 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 700 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.638 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839315+0000", 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839315+0000", 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839318+0000", 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.286e-06 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839333+0000", 2024-04-18T23:12:16.639 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4970000000000001e-05 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839370+0000", 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.7311999999999998e-05 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839395+0000", 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5026000000000001e-05 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839543+0000", 2024-04-18T23:12:16.640 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014795699999999999 2024-04-18T23:12:16.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839731+0000", 2024-04-18T23:12:16.641 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018835799999999999 2024-04-18T23:12:16.641 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.641 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.641 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.641 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.841353+0000", 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0016218070000000001 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.846909+0000", 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0055556499999999997 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.447943+0000", 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.60103379700000004 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.642 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.448087+0000", 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00014405700000000001 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:701 3.5s0 3:a6fe874e:::smithi14230322-33:head [rollback 1b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.839449+0000", 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.779838675000001, 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.72747437500000001, 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.643 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 701 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839449+0000", 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839449+0000", 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.644 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839450+0000", 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.677e-06 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839460+0000", 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.5030000000000003e-06 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839481+0000", 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1359999999999999e-05 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.645 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839499+0000", 2024-04-18T23:12:16.646 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7549000000000001e-05 2024-04-18T23:12:16.646 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.646 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.646 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.646 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.842939+0000", 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.003440425 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.843157+0000", 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021817200000000001 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.844478+0000", 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001320697 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.846923+0000", 2024-04-18T23:12:16.647 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024451770000000002 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566709+0000", 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71978546300000001 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.566923+0000", 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00021435200000000001 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:702 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~0] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:16.648 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.839557+0000", 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.779730106999999, 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.72777091400000005, 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 702 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.649 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839557+0000", 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839557+0000", 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839559+0000", 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.24e-06 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.650 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839565+0000", 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.5830000000000003e-06 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839586+0000", 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0455999999999999e-05 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.839608+0000", 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2578999999999998e-05 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.651 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.845588+0000", 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0059799830000000003 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.845742+0000", 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000154096 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.846498+0000", 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00075576500000000004 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.847239+0000", 2024-04-18T23:12:16.652 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00074123500000000003 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.567154+0000", 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71991484800000005 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.567328+0000", 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000174129 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.653 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:716 3.5s0 3:a832c25e:::smithi14230322-46:head [append 0~753664] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.944077+0000", 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.675210961000001, 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.63339562400000005, 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.654 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 716 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.944077+0000", 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.944077+0000", 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.655 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.944079+0000", 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1770000000000001e-06 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945290+0000", 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0012111699999999999 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945320+0000", 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.0173000000000001e-05 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.656 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945335+0000", 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4909000000000001e-05 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945516+0000", 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018068899999999999 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945705+0000", 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000189829 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.947796+0000", 2024-04-18T23:12:16.657 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002090728 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.952820+0000", 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0050237529999999997 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.577336+0000", 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.624516352 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.577472+0000", 2024-04-18T23:12:16.658 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013584400000000001 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:717 3.5s0 3:a832c25e:::smithi14230322-46:head [append 0~688128] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.945360+0000", 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.673927012, 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.63253810799999999, 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.659 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 717 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945360+0000", 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945360+0000", 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.660 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.945362+0000", 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.2860000000000001e-06 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946063+0000", 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00070088599999999996 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946077+0000", 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4793999999999999e-05 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.661 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946087+0000", 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.7759999999999992e-06 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.949388+0000", 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033004850000000001 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.949576+0000", 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000188412 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.662 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.951394+0000", 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001817962 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.055973+0000", 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.10457884000000001 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.577726+0000", 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.52175303799999995 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.577899+0000", 2024-04-18T23:12:16.663 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017262899999999999 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:718 3.5s0 3:a832c25e:::smithi14230322-46:head [append 0~475136] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:49.946109+0000", 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.673178511, 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.63219438400000005, 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 718 2024-04-18T23:12:16.664 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946109+0000", 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946109+0000", 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946110+0000", 2024-04-18T23:12:16.665 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.540000000000001e-07 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946595+0000", 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00048526400000000001 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:49.946611+0000", 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.6099999999999998e-05 2024-04-18T23:12:16.666 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046764+0000", 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100152771 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.053013+0000", 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0062488439999999999 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.053213+0000", 2024-04-18T23:12:16.667 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00020024299999999999 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.054693+0000", 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001479408 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.056313+0000", 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001619948 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.578170+0000", 2024-04-18T23:12:16.668 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.521857389 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.578303+0000", 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013346300000000001 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:719 3.5s0 3:a832c25e:::smithi14230322-46:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:50.046818+0000", 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.572469287000001, 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.63283090600000003, 2024-04-18T23:12:16.669 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 719 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046818+0000", 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.670 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046818+0000", 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046820+0000", 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.1900000000000002e-06 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046838+0000", 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7379000000000001e-05 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.671 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.046864+0000", 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5803e-05 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147013+0000", 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100149482 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147193+0000", 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000180309 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.672 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147532+0000", 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00033901800000000001 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.148946+0000", 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014138499999999999 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.447622+0000", 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.298675886 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.673 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.679477+0000", 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.23185497699999999 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.679649+0000", 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00017201200000000001 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.674 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:720 3.5s0 3:a832c25e:::smithi14230322-46:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:50.147067+0000", 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 26.472220927999999, 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.91909054300000004, 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 720 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147067+0000", 2024-04-18T23:12:16.675 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147067+0000", 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147069+0000", 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.063e-06 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147081+0000", 2024-04-18T23:12:16.676 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.19e-05 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147103+0000", 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2436e-05 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.147120+0000", 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7010000000000001e-05 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.149640+0000", 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0025200600000000002 2024-04-18T23:12:16.677 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.149905+0000", 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026456 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.679775+0000", 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.52987033900000002 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:50.679978+0000", 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000202812 2024-04-18T23:12:16.678 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:51.066157+0000", 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.386179363 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:796 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~147456] snapc 1d=[1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:54.705423+0000", 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.913864875000002, 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50985007800000004, 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.679 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 796 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705423+0000", 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705423+0000", 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.680 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705424+0000", 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1260000000000001e-06 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705559+0000", 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013499399999999999 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.681 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705576+0000", 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7465999999999998e-05 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705584+0000", 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.0399999999999993e-06 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.008071+0000", 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30248644499999999 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.682 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.008371+0000", 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029988999999999998 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.010226+0000", 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001855754 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013512+0000", 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0032859149999999999 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.683 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.215049+0000", 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20153676700000001 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.215273+0000", 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022368099999999999 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:797 3.5s0 3:a6fe874e:::smithi14230322-33:head [setxattr _header (57) in=7b] snapc 1d=[1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e88)", 2024-04-18T23:12:16.684 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:54.705597+0000", 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.913690293999998, 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61118310200000003, 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 797 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705597+0000", 2024-04-18T23:12:16.685 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705597+0000", 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705598+0000", 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.9299999999999997e-07 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.686 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705601+0000", 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.332e-06 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705607+0000", 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 6.2230000000000004e-06 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705613+0000", 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 5.13e-06 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.687 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.011256+0000", 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30564371299999998 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.011440+0000", 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018343 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.012418+0000", 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00097794900000000009 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.688 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.013799+0000", 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001381506 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.316552+0000", 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.302752888 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.316780+0000", 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00022823799999999999 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.689 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:798 3.5s0 3:a6fe874e:::smithi14230322-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e88)", 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:11:54.705619+0000", 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 21.913668453, 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.71576949899999998, 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.690 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 798 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705619+0000", 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705619+0000", 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.691 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705619+0000", 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.1300000000000001e-07 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705622+0000", 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0870000000000001e-06 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705629+0000", 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.418e-06 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.692 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:54.705634+0000", 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 4.6550000000000003e-06 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.317848+0000", 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.61221415199999996 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.318138+0000", 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00029039700000000002 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.693 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:11:55.421389+0000", 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.103250377 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:954 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~737280] snapc 25=[25,24,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.621433+0000", 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.9978545380000003, 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.37044549900000001, 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.694 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 954 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.621433+0000", 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.695 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.621433+0000", 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.621436+0000", 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.4829999999999999e-06 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.622390+0000", 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00095352299999999996 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.696 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.622423+0000", 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.2916999999999997e-05 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.622438+0000", 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.4979e-05 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.622571+0000", 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00013285099999999999 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.697 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.622753+0000", 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000182704 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.625210+0000", 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024570030000000001 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.631919+0000", 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0067089970000000004 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.698 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.991715+0000", 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.359795435 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.991878+0000", 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016360699999999999 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:955 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~770048] snapc 25=[25,24,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.626144+0000", 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.9931437509999999, 2024-04-18T23:12:16.699 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50887431400000005, 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 955 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.626144+0000", 2024-04-18T23:12:16.700 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.626144+0000", 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.626147+0000", 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.1599999999999998e-06 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.627282+0000", 2024-04-18T23:12:16.701 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0011355530000000001 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.627321+0000", 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 3.8418999999999999e-05 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.627344+0000", 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.2785999999999998e-05 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.628391+0000", 2024-04-18T23:12:16.702 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010476649999999999 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.628556+0000", 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00016445399999999999 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.630436+0000", 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0018804099999999999 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.703 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.634924+0000", 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0044875699999999998 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.134894+0000", 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49997039300000001 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.135018+0000", 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012390400000000001 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.704 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:956 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~540672] snapc 25=[25,24,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.628846+0000", 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.9904410059999993, 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50644018599999996, 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 956 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.705 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.628846+0000", 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.628846+0000", 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.628849+0000", 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.5629999999999999e-06 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.706 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.629574+0000", 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000724941 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.629601+0000", 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7461999999999999e-05 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.629622+0000", 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.0353e-05 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.707 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.632032+0000", 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0024100969999999999 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.632218+0000", 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018624399999999999 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.633840+0000", 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0016214809999999999 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.708 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.635254+0000", 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0014140019999999999 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.135197+0000", 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.49994357 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.135287+0000", 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.9473000000000002e-05 2024-04-18T23:12:16.709 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:957 3.5s0 3:a6fe874e:::smithi14230322-33:head [append 0~245760] snapc 25=[25,24,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.630228+0000", 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.9890597680000006, 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.50537000600000004, 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.710 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 957 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.630228+0000", 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.630228+0000", 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.711 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.630230+0000", 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7039999999999999e-06 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.630590+0000", 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00035992700000000001 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.630618+0000", 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.7339e-05 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.712 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.730703+0000", 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100085459 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.730853+0000", 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00015025699999999999 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.731034+0000", 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018034 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.713 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.732115+0000", 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0010815359999999999 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.733261+0000", 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.001145917 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.135505+0000", 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40224407899999998 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.714 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.135598+0000", 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 9.2448000000000007e-05 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:958 3.5s0 3:a6fe874e:::smithi14230322-33:head [setxattr _header (57) in=7b] snapc 25=[25,24,22,21,20,1d,1c,1b,19,15,13] ondisk+write+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.730728+0000", 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.8885589920000001, 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.40553212100000002, 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.715 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 958 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.730728+0000", 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.730728+0000", 2024-04-18T23:12:16.716 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.730730+0000", 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.3319999999999999e-06 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.730737+0000", 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 7.25e-06 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.730748+0000", 2024-04-18T23:12:16.717 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.0879999999999999e-05 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.830839+0000", 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.100090875 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.831109+0000", 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00026968800000000002 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.718 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.831295+0000", 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00018626100000000001 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_started", 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.832270+0000", 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00097566899999999997 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "sub_op_committed", 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.834432+0000", 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.002161517 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "commit_sent", 2024-04-18T23:12:16.719 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.136150+0000", 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.30171755700000003 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.136261+0000", 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000111092 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: "description": "osd_op(client.4568.0:959 3.5s0 3:a6fe874e:::smithi14230322-33:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: "initiated_at": "2024-04-18T23:12:06.830878+0000", 2024-04-18T23:12:16.720 INFO:teuthology.orchestra.run.smithi142.stdout: "age": 9.7884093229999998, 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.51441819499999997, 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "continuous": false, 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "type_data": { 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "flag_point": "started", 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "client_info": { 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "client": "client.4568", 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "tid": 959 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "events": [ 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "initiated", 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.830878+0000", 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.721 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "header_read", 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.830878+0000", 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "throttled", 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.830880+0000", 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.7740000000000001e-06 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "all_read", 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.830889+0000", 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 8.7299999999999994e-06 2024-04-18T23:12:16.722 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "dispatched", 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.830909+0000", 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 2.035e-05 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "queued_for_pg", 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.830921+0000", 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 1.1868999999999999e-05 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.834266+0000", 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.0033453269999999999 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.723 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:06.834374+0000", 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.000107404 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "reached_pg", 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.136804+0000", 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.302429954 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "started", 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.136932+0000", 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.00012862799999999999 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: }, 2024-04-18T23:12:16.724 INFO:teuthology.orchestra.run.smithi142.stdout: { 2024-04-18T23:12:16.725 INFO:teuthology.orchestra.run.smithi142.stdout: "event": "done", 2024-04-18T23:12:16.725 INFO:teuthology.orchestra.run.smithi142.stdout: "time": "2024-04-18T23:12:07.345296+0000", 2024-04-18T23:12:16.725 INFO:teuthology.orchestra.run.smithi142.stdout: "duration": 0.20836415899999999 2024-04-18T23:12:16.725 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.725 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.725 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.725 INFO:teuthology.orchestra.run.smithi142.stdout: } 2024-04-18T23:12:16.725 INFO:teuthology.orchestra.run.smithi142.stdout: ] 2024-04-18T23:12:16.725 INFO:teuthology.orchestra.run.smithi142.stdout:} 2024-04-18T23:12:16.727 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.1 to 1 (10000) 2024-04-18T23:12:16.727 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-18T23:12:16.728 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-18T23:12:16.728 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:16.724+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:16.827 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:16.822+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:16.897 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:16.897 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:12:16.897 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:12:16.898 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:16.913 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-18T23:12:16.928 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:16.927+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:17.029 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:17.027+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:17.075 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:17.075 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:12:17.075 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:12:17.075 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:12:17.075 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:17.089 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-18T23:12:17.136 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:12:17.134+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:12:17.214 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:17.214 INFO:teuthology.orchestra.run.smithi073.stdout: "size": 20, 2024-04-18T23:12:17.214 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 600, 2024-04-18T23:12:17.214 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [ 2024-04-18T23:12:17.214 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.214 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:417 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~524288 in=524288b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:17.214 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.690739852999997, 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30835631899999999, 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 417 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.215 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520103+0000", 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520104+0000", 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2389999999999999e-06 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.216 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520670+0000", 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00056593599999999996 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520690+0000", 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9945e-05 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.520699+0000", 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0429999999999996e-06 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.217 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618217+0000", 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.097518176999999998 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.618227+0000", 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7860000000000008e-06 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622364+0000", 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0041371389999999997 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.218 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.622653+0000", 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00028858600000000002 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.624345+0000", 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001691782 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.727165+0000", 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102820087 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828093+0000", 2024-04-18T23:12:17.219 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100928079 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828459+0000", 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00036652000000000001 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:418 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~245760 in=245760b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.689256997000001, 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30828365299999999, 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.220 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 418 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.221 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521586+0000", 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521587+0000", 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2100000000000001e-06 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521960+0000", 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000373402 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.222 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521972+0000", 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.207e-05 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521979+0000", 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.7839999999999998e-06 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725251+0000", 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20327163200000001 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.223 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.725428+0000", 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000177071 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.726423+0000", 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00099555200000000003 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828646+0000", 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102223007 2024-04-18T23:12:17.224 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829746+0000", 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0010997509999999999 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829869+0000", 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00012317400000000001 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.225 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:419 3.6s0 3:635103ff:::smithi14230322-31:head [setxattr _header (57) in=7b] snapc f=[f,d,c,b,a,9,7] ondisk+write+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.688851980000003, 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.71019062499999996, 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 419 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.226 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521991+0000", 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2200000000000004e-07 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.227 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521994+0000", 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.8289999999999999e-06 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.521999+0000", 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.8470000000000003e-06 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522003+0000", 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1250000000000003e-06 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.228 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828718+0000", 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30671521899999998 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.828796+0000", 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.7502000000000004e-05 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829209+0000", 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00041271100000000001 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829944+0000", 2024-04-18T23:12:17.229 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00073506600000000004 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.231940+0000", 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.401996146 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232181+0000", 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024165800000000001 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.230 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:420 3.6s0 3:635103ff:::smithi14230322-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e56)", 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 55.688834307, 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.108457993, 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 420 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.231 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522008+0000", 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522009+0000", 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4999999999999998e-07 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522012+0000", 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.819e-06 2024-04-18T23:12:17.232 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522017+0000", 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.2100000000000001e-06 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.522021+0000", 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.1400000000000002e-06 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829499+0000", 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30747766399999998 2024-04-18T23:12:17.233 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:21.829621+0000", 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000122502 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232655+0000", 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.403033435 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.232821+0000", 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000166279 2024-04-18T23:12:17.234 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:22.630466+0000", 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.39764559399999999 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:721 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~770048] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.882464+0000", 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.328378263000001, 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.37029923799999998, 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.235 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 721 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.882464+0000", 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.882464+0000", 2024-04-18T23:12:17.236 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.882467+0000", 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.6460000000000002e-06 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883686+0000", 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0012187159999999999 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883718+0000", 2024-04-18T23:12:17.237 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2186e-05 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883733+0000", 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4970000000000001e-05 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883823+0000", 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0488999999999994e-05 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.883966+0000", 2024-04-18T23:12:17.238 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014226500000000001 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.885426+0000", 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0014602409999999999 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.890445+0000", 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0050192099999999996 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.252676+0000", 2024-04-18T23:12:17.239 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.36223060800000001 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.252764+0000", 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7906999999999997e-05 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:722 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~704512] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.887144+0000", 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.323698801999999, 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38817684699999999, 2024-04-18T23:12:17.240 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 722 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.887144+0000", 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.887144+0000", 2024-04-18T23:12:17.241 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.887153+0000", 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.0799999999999995e-06 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888445+0000", 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00129243 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.242 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888476+0000", 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0586000000000002e-05 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888492+0000", 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5933000000000001e-05 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888562+0000", 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9862000000000003e-05 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.888675+0000", 2024-04-18T23:12:17.243 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000113402 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.889711+0000", 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001035869 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.995104+0000", 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.105393192 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275227+0000", 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.28012296199999998 2024-04-18T23:12:17.244 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275321+0000", 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3530999999999996e-05 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:723 3.6s0 3:635103ff:::smithi14230322-31:head [append 0~458752] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.890673+0000", 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.320169829000001, 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38502245800000001, 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.245 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 723 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.890673+0000", 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.246 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.890673+0000", 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.890675+0000", 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.753e-06 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891295+0000", 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00062035499999999995 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891321+0000", 2024-04-18T23:12:17.247 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.5469e-05 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891333+0000", 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.2816999999999999e-05 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891402+0000", 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9068999999999999e-05 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891521+0000", 2024-04-18T23:12:17.248 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000118157 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.892362+0000", 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00084165300000000004 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.995106+0000", 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.102743341 2024-04-18T23:12:17.249 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275555+0000", 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.28044971400000002 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275695+0000", 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00014013 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.250 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:724 3.6s0 3:635103ff:::smithi14230322-31:head [setxattr _header (57) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891361+0000", 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.319481706000001, 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38458247699999998, 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 724 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.251 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891361+0000", 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891361+0000", 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891362+0000", 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.387e-06 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.252 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891370+0000", 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.2860000000000002e-06 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891389+0000", 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9108999999999999e-05 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891404+0000", 2024-04-18T23:12:17.253 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.5024e-05 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.892942+0000", 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.001537957 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.893044+0000", 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00010249799999999999 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.254 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.893555+0000", 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00051112599999999998 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.995108+0000", 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101552937 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "commit_sent", 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275846+0000", 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.28073780100000001 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.255 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.275944+0000", 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.7351999999999999e-05 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:725 3.6s0 3:635103ff:::smithi14230322-31:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891424+0000", 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.319418788, 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.68865348800000004, 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:17.256 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 725 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891424+0000", 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891424+0000", 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.257 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891426+0000", 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7370000000000001e-06 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891433+0000", 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.3459999999999996e-06 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891452+0000", 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8972999999999999e-05 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.258 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891466+0000", 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.3852999999999999e-05 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.893797+0000", 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0023310200000000001 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.893902+0000", 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000104888 2024-04-18T23:12:17.259 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.276479+0000", 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38257698400000001 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.276775+0000", 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.000296228 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.580077+0000", 2024-04-18T23:12:17.260 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30330245900000002 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:726 3.6s0 3:6db21afe:::smithi14230322-12:15 [read 0~131072,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891478+0000", 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.319364365999999, 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38608822199999998, 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.261 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 726 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891478+0000", 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891478+0000", 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.262 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891479+0000", 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9699999999999995e-07 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891484+0000", 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.7469999999999996e-06 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891493+0000", 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3160000000000003e-06 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.263 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891500+0000", 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.1860000000000003e-06 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.894032+0000", 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0025314819999999998 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.894127+0000", 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.4858000000000006e-05 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.264 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.277567+0000", 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38343993599999998 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:727 3.6s0 3:6db21afe:::smithi14230322-12:15 [read 0~131072,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891510+0000", 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.319333022999999, 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.48656820899999997, 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.265 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 727 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891510+0000", 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891510+0000", 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.266 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891510+0000", 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0299999999999999e-07 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891514+0000", 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4070000000000002e-06 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891522+0000", 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.6459999999999999e-06 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.267 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891529+0000", 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.8929999999999999e-06 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.994525+0000", 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10299591600000001 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.994591+0000", 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.6243999999999998e-05 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.378078+0000", 2024-04-18T23:12:17.268 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.38348650000000001 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:728 3.6s0 3:6db21afe:::smithi14230322-12:15 [read 0~131072,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.891537+0000", 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.319305651000001, 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.58775005700000005, 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.269 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 728 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891537+0000", 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891537+0000", 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.270 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891538+0000", 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.9999999999999997e-07 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891542+0000", 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.9330000000000003e-06 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891550+0000", 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.2870000000000004e-06 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.891557+0000", 2024-04-18T23:12:17.271 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.9870000000000002e-06 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.994791+0000", 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10323368600000001 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.994865+0000", 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.4899000000000002e-05 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.479287+0000", 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.484421665 2024-04-18T23:12:17.272 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.5s5 83/72 ECSubWrite(tid=39, reqid=client.4568.0:701, at_version=83'87, trim_to=0'0, roll_forward_to=82'84))", 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.990703+0000", 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.220139915000001, 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30279335000000002, 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990703+0000", 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.273 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990703+0000", 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990703+0000", 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.5300000000000004e-07 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990722+0000", 2024-04-18T23:12:17.274 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.8445999999999999e-05 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990737+0000", 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.47e-05 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990748+0000", 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.1756000000000001e-05 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.090937+0000", 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10018898900000001 2024-04-18T23:12:17.275 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.090953+0000", 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6079999999999999e-05 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293414+0000", 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20246087500000001 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293496+0000", 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.1850999999999999e-05 2024-04-18T23:12:17.276 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.5s5 83/72 ECSubWrite(tid=40, reqid=client.4568.0:702, at_version=83'88, trim_to=0'0, roll_forward_to=82'84))", 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:49.990770+0000", 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.220072287000001, 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.30279787000000002, 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.277 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990770+0000", 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990770+0000", 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990771+0000", 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.3000000000000001e-07 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.278 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990803+0000", 2024-04-18T23:12:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2353e-05 2024-04-18T23:12:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990820+0000", 2024-04-18T23:12:17.279 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.6929e-05 2024-04-18T23:12:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:49.990829+0000", 2024-04-18T23:12:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.7600000000000008e-06 2024-04-18T23:12:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.281 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.091673+0000", 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.100843842 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.091704+0000", 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.1582999999999999e-05 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293505+0000", 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20180018199999999 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.282 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293568+0000", 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.3690999999999997e-05 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.ds4 83/31 ECSubWrite(tid=142, reqid=client.4568.0:705, at_version=83'120, trim_to=0'0, roll_forward_to=83'116))", 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.060665+0000", 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.150177319000001, 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.232972237, 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.283 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.060665+0000", 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.060665+0000", 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.060667+0000", 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.933e-06 2024-04-18T23:12:17.284 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.061042+0000", 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00037508399999999998 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.061083+0000", 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.0286e-05 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.061119+0000", 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.6001999999999999e-05 2024-04-18T23:12:17.285 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.091613+0000", 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.03049437 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.091637+0000", 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4162e-05 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293573+0000", 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20193602499999999 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.286 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.293638+0000", 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.4375000000000006e-05 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpReadReply(3.6s0 83/72 ECSubReadReply(tid=56, attrs_read=0))", 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 27.014246067999999, 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.28271478999999999, 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.287 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.288 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.196597+0000", 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.296432+0000", 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.099835462 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.396590+0000", 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.10015741 2024-04-18T23:12:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.478986+0000", 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.082396805000000004 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:11:50.479311+0000", 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00032511299999999998 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.290 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "MOSDECSubOpWrite(3.bs1 94/31 ECSubWrite(tid=415, reqid=client.4568.0:833, at_version=0'0, trim_to=0'0, roll_forward_to=94'175))", 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:01.479134+0000", 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 15.73170831, 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.301403798, 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "reached pg", 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.479134+0000", 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.479134+0000", 2024-04-18T23:12:17.291 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.479136+0000", 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.485e-06 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.479153+0000", 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7258999999999999e-05 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.479174+0000", 2024-04-18T23:12:17.292 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.0834000000000001e-05 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.479184+0000", 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.0193999999999999e-05 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.679473+0000", 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.20028852999999999 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_started", 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.679487+0000", 2024-04-18T23:12:17.293 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.4615e-05 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "sub_op_committed", 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.780508+0000", 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.101020822 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:01.780538+0000", 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.0059000000000001e-05 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.294 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:960 3.6s0 3:635103ff:::smithi14230322-31:head [read 0~1933312,checksum,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:06.629162+0000", 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.581680355, 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40539902500000002, 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 960 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.295 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629162+0000", 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629162+0000", 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629164+0000", 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.9240000000000001e-06 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.296 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629173+0000", 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.4970000000000001e-06 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629191+0000", 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 1.7944e-05 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629200+0000", 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 9.3549999999999994e-06 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.297 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629283+0000", 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 8.3244999999999993e-05 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629531+0000", 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00024791400000000001 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.034561+0000", 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40503014599999998 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.298 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:961 3.6s0 3:635103ff:::smithi14230322-31:head [read 0~1933312,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:06.629222+0000", 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.581621113000001, 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40673737300000001, 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 961 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.299 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629222+0000", 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629222+0000", 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629222+0000", 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0299999999999999e-07 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.300 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629226+0000", 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.2760000000000001e-06 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629232+0000", 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.0750000000000003e-06 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629237+0000", 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 5.5060000000000003e-06 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629930+0000", 2024-04-18T23:12:17.301 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00069260700000000003 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.630009+0000", 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 7.9542000000000005e-05 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.035959+0000", 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40594976399999999 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.302 INFO:teuthology.orchestra.run.smithi073.stdout: "description": "osd_op(client.4568.0:962 3.6s0 3:635103ff:::smithi14230322-31:head [read 0~1933312,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e100)", 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "initiated_at": "2024-04-18T23:12:06.629246+0000", 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "age": 10.581596771999999, 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.40911155799999999, 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "continuous": false, 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "type_data": { 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "flag_point": "started", 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "client_info": { 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "client": "client.4568", 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "tid": 962 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "events": [ 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "initiated", 2024-04-18T23:12:17.303 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629246+0000", 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "header_read", 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629246+0000", 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "throttled", 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629246+0000", 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 3.4200000000000002e-07 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "all_read", 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629249+0000", 2024-04-18T23:12:17.304 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 2.4140000000000001e-06 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "dispatched", 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629255+0000", 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 6.1179999999999997e-06 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "queued_for_pg", 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.629259+0000", 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 4.233e-06 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "reached_pg", 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.630439+0000", 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.0011803339999999999 2024-04-18T23:12:17.305 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "started", 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:06.630602+0000", 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.00016258599999999999 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: { 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "event": "done", 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "time": "2024-04-18T23:12:07.038358+0000", 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: "duration": 0.407755531 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: } 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2024-04-18T23:12:17.306 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:17.308 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-18T23:12:17.309 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:17.305+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:17.409 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:17.405+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:17.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:12:17.481 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:17.481 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:12:17.481 INFO:teuthology.orchestra.run.smithi073.stdout: "num_ops": 0 2024-04-18T23:12:17.481 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:17.496 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-18T23:12:17.511 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:17.505+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:17.589 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:12:17.589 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:12:17.609 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:12:17.610 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:12:17.611 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:17.606+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:17.631 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:12:17.631 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:12:17.647 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.2 to 1 (10000) 2024-04-18T23:12:17.652 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:12:17.653 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:12:17.666 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:12:17.667 INFO:teuthology.orchestra.run.smithi073.stdout: "ops": [], 2024-04-18T23:12:17.667 INFO:teuthology.orchestra.run.smithi073.stdout: "complaint_time": 30, 2024-04-18T23:12:17.667 INFO:teuthology.orchestra.run.smithi073.stdout: "num_blocked_ops": 0 2024-04-18T23:12:17.667 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:12:17.670 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-18T23:12:17.671 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:12:17.671 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:12:17.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-18T23:12:17.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-18T23:12:17.692 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:12:17.692 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:12:17.712 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:17.709+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:17.728 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:12:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:12:17.764 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:12:17.764 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:12:17.788 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:12:17.789 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:12:17.809 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:12:17.810 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:12:17.812 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:17.813 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:17.813 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:17.811+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:17.829 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_ops_in_flight 2024-04-18T23:12:17.833 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:12:17.834 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:12:17.854 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:12:17.855 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:12:17.873 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:12:17.873 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:12:17.893 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:12:17.893 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:12:17.914 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:12:17.914 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:17.909+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:17.914 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:12:17.934 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:12:17.935 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:12:17.947 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:17.947 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:17.961 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_blocked_ops 2024-04-18T23:12:18.048 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:18.046+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:18.109 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:18.109 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:18.116 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:18.113+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:18.122 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.8.asok dump_historic_ops 2024-04-18T23:12:18.217 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:18.213+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:18.277 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:18.277 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:18.290 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_ops_in_flight 2024-04-18T23:12:18.318 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:18.319+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:18.404 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:18.404 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:18.418 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_blocked_ops 2024-04-18T23:12:18.419 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:18.417+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:18.519 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is set 2024-04-18T23:12:18.520 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.3 to 1 (10000) 2024-04-18T23:12:18.520 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:18.517+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:18.544 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:18.544 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:18.547 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-18T23:12:18.558 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.12.asok dump_historic_ops 2024-04-18T23:12:18.620 INFO:tasks.ceph.osd.9.smithi120.stderr:2024-04-18T23:12:18.617+0000 7f604fafe640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 9 (PID: 25058) UID: 0 2024-04-18T23:12:18.715 INFO:teuthology.orchestra.run.smithi073.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:18.716 INFO:teuthology.orchestra.run.smithi073.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:18.721 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:18.720+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:18.731 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-18T23:12:18.822 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:18.817+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:18.865 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:18.865 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:18.880 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-18T23:12:18.922 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:18.920+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:19.023 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:19.022+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:19.049 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:19.050 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:19.063 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-18T23:12:19.124 INFO:tasks.ceph.osd.3.smithi142.stderr:2024-04-18T23:12:19.123+0000 7fc0727cd640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 25337) UID: 0 2024-04-18T23:12:19.186 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:19.186 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:19.200 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-18T23:12:19.225 INFO:tasks.ceph.osd.8.smithi073.stderr:2024-04-18T23:12:19.222+0000 7f615ea00640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 8 (PID: 25177) UID: 0 2024-04-18T23:12:19.320 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:19.321 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:19.326 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:19.321+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:19.335 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-18T23:12:19.427 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:12:19.428+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:12:19.497 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:19.497 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:19.512 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-18T23:12:19.528 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:19.527+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:19.629 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:19.625+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:19.629 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:19.643 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_ops_in_flight 2024-04-18T23:12:19.670 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.4 to 1 (10000) 2024-04-18T23:12:19.693 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-18T23:12:19.731 INFO:tasks.ceph.osd.5.smithi120.stderr:2024-04-18T23:12:19.729+0000 7f2e52e0a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 25056) UID: 0 2024-04-18T23:12:19.813 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:19.813 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:19.827 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_blocked_ops 2024-04-18T23:12:19.831 INFO:tasks.ceph.osd.0.smithi073.stderr:2024-04-18T23:12:19.826+0000 7fd5966d2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 25173) UID: 0 2024-04-18T23:12:19.932 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:19.932+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:19.957 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-18T23:12:19.995 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:19.995 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:20.010 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.9.asok dump_historic_ops 2024-04-18T23:12:20.033 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:12:20.032+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:12:20.102 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:12:20.102 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:12:20.120 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:12:20.121 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:12:20.132 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:20.132 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:20.133 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:12:20.132+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:12:20.140 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:12:20.140 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:12:20.146 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_ops_in_flight 2024-04-18T23:12:20.161 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:12:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:12:20.178 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:12:20.179 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:12:20.195 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:12:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:12:20.215 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:12:20.215 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:12:20.234 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:20.235+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:20.235 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:12:20.236 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:12:20.253 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:12:20.253 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:12:20.264 INFO:teuthology.orchestra.run.smithi120.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:20.264 INFO:teuthology.orchestra.run.smithi120.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:20.273 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:12:20.273 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:12:20.278 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_blocked_ops 2024-04-18T23:12:20.294 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:12:20.294 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:12:20.316 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:12:20.316 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:12:20.334 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:12:20.334 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:12:20.335 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:20.332+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:20.354 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:12:20.354 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:12:20.375 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:12:20.375 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:12:20.395 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:12:20.396 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:12:20.400 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [], 2024-04-18T23:12:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "complaint_time": 30, 2024-04-18T23:12:20.400 INFO:teuthology.orchestra.run.smithi120.stdout: "num_blocked_ops": 0 2024-04-18T23:12:20.400 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:20.414 DEBUG:teuthology.orchestra.run.smithi120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.13.asok dump_historic_ops 2024-04-18T23:12:20.436 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:20.436+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout:{ 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: "size": 20, 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 600, 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: "ops": [ 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:114 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.936913907000005, 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62069844100000005, 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.532 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 114 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.533 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594564+0000", 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594565+0000", 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594570+0000", 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.186e-06 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.534 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594582+0000", 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1763000000000001e-05 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594590+0000", 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.5290000000000003e-06 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010572+0000", 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.415982252 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.535 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010727+0000", 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015558400000000001 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.010971+0000", 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000243339 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.015929+0000", 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0049583639999999998 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.536 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215108+0000", 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19917934400000001 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215199+0000", 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0556000000000005e-05 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215263+0000", 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.3874000000000005e-05 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.537 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.936877580000001, 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62089894099999998, 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:12:20.538 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 115 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.539 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594601+0000", 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.1999999999999999e-07 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594746+0000", 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014446700000000001 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594761+0000", 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4792e-05 2024-04-18T23:12:20.540 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594769+0000", 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.6160000000000002e-06 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013742+0000", 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.418972438 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.013974+0000", 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00023265699999999999 2024-04-18T23:12:20.541 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.014328+0000", 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00035345 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018069+0000", 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0037411079999999999 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215346+0000", 2024-04-18T23:12:20.542 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19727757900000001 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215435+0000", 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8245999999999995e-05 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.215500+0000", 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.4968000000000005e-05 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.543 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.936694329000005, 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.72133309899999998, 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 116 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.544 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594784+0000", 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.545 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594785+0000", 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9599999999999998e-07 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594790+0000", 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.4229999999999996e-06 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594803+0000", 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2196e-05 2024-04-18T23:12:20.546 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594810+0000", 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.6190000000000002e-06 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016062+0000", 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.42125235 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.547 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016215+0000", 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000152051 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016420+0000", 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000205916 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.018082+0000", 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001661718 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.548 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.315968+0000", 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29788616600000001 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316051+0000", 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.2897000000000003e-05 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316117+0000", 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5866999999999998e-05 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.549 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4260.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27)", 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 92.936656696, 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.721588585, 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4260", 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.73:0/2838686474", 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 117 2024-04-18T23:12:20.550 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.551 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594822+0000", 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5000000000000002e-07 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594829+0000", 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.7830000000000001e-06 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.594841+0000", 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1505000000000001e-05 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.552 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:47.694940+0000", 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100099585 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.016967+0000", 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.32202692799999999 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017224+0000", 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000256548 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for subops from 2", 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.017583+0000", 2024-04-18T23:12:20.553 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000359689 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "op_commit", 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.020139+0000", 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0025556649999999999 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_commit_rec", 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316256+0000", 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.29611688800000002 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316345+0000", 2024-04-18T23:12:20.554 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.9285999999999999e-05 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:48.316410+0000", 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 6.5058e-05 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:26 3.2s0 3:4250b374:::smithi14230322-9:head [setxattr _header (55) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 84.007270669999997, 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60852922899999995, 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.555 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 26 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524208+0000", 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.556 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524210+0000", 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.996e-06 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524220+0000", 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.0056e-05 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.524242+0000", 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2212e-05 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.557 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.525734+0000", 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001491783 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528099+0000", 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0023654380000000001 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.528318+0000", 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000219065 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.558 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.529513+0000", 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011952530000000001 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636414+0000", 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.106900906 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132457+0000", 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49604301899999997 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.132737+0000", 2024-04-18T23:12:20.559 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00027950099999999998 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:31 3.2s0 3:42c34e84:::smithi14230322-11:head [append 0~180224,stat] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 84.005156245999999, 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60793644899999999, 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.560 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 31 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526322+0000", 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.561 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526324+0000", 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9400000000000001e-06 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526836+0000", 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00051192200000000005 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526869+0000", 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2978000000000002e-05 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.562 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526890+0000", 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1481999999999999e-05 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531167+0000", 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042762490000000002 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.531508+0000", 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034109400000000001 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.533620+0000", 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0021122279999999998 2024-04-18T23:12:20.563 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636418+0000", 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102798509 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134075+0000", 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.497656453 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.134259+0000", 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000183594 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.564 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:32 3.2s0 3:42c34e84:::smithi14230322-11:head [setxattr _header (56) in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 84.004553336000001, 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60874687999999999, 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 32 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.565 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526925+0000", 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526927+0000", 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0389999999999999e-06 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.566 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526936+0000", 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.2599999999999994e-06 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526957+0000", 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0302e-05 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.526973+0000", 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.6359e-05 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634473+0000", 2024-04-18T23:12:20.567 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.107499583 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.634639+0000", 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016687200000000001 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.635580+0000", 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00094034999999999995 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:56.636830+0000", 2024-04-18T23:12:20.568 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0012506220000000001 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135509+0000", 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.49867835399999999 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:10:57.135672+0000", 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000163139 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.569 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:618 3.2s0 3:42c34e84:::smithi14230322-11:head [read 0~1835008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.909234419000001, 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.63639373399999999, 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 618 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.570 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622244+0000", 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622245+0000", 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3799999999999999e-06 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.571 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622253+0000", 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 7.4889999999999998e-06 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.622276+0000", 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2938999999999998e-05 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722418+0000", 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100141811 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822632+0000", 2024-04-18T23:12:20.572 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100214391 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822645+0000", 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2863e-05 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.841571+0000", 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.018926235 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.841706+0000", 2024-04-18T23:12:20.573 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000134528 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.258638+0000", 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.41693209799999997 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:619 3.2s0 3:42c34e84:::smithi14230322-11:head [read 0~1835008,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.809018858999998, 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.94106379600000001, 2024-04-18T23:12:20.574 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 619 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722459+0000", 2024-04-18T23:12:20.575 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722462+0000", 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.12e-06 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722475+0000", 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3475999999999999e-05 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722500+0000", 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.4817999999999999e-05 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.576 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.722517+0000", 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7452000000000002e-05 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822716+0000", 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100198302 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.822729+0000", 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3036e-05 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.577 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.842152+0000", 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.019423458000000001 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.842286+0000", 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013423100000000001 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.663523+0000", 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.82123690299999996 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.578 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:628 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~589824] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.692363468000003, 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.82267251500000005, 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 628 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:12:20.579 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839115+0000", 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.839118+0000", 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.6759999999999999e-06 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840018+0000", 2024-04-18T23:12:20.580 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00090027100000000001 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840051+0000", 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.3426999999999997e-05 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.840069+0000", 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7869000000000001e-05 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.949596+0000", 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10952640800000001 2024-04-18T23:12:20.581 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.949748+0000", 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0001526 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.951307+0000", 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001558633 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.052856+0000", 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.101548762 2024-04-18T23:12:20.582 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.661580+0000", 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.608724088 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.661787+0000", 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00020778100000000001 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.583 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:629 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~540672] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.684733967, 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91730556200000002, 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 629 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.584 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846744+0000", 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.846746+0000", 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5799999999999999e-06 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847461+0000", 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00071526900000000004 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.585 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847485+0000", 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.3509000000000001e-05 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.847497+0000", 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.2675e-05 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.047393+0000", 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19989558700000001 2024-04-18T23:12:20.586 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.047586+0000", 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000192568 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.049081+0000", 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001495225 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.052942+0000", 2024-04-18T23:12:20.587 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0038612070000000002 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.763898+0000", 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.71095591800000002 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764050+0000", 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015202400000000001 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.588 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:630 3.2s0 3:4b92d38e:::smithi14230322-19:head [append 0~622592] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.675871981999997, 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.908871761, 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 630 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.589 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855606+0000", 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.855609+0000", 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.17e-06 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.590 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856431+0000", 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00082242899999999996 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856463+0000", 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.2320999999999998e-05 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856479+0000", 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5396e-05 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.050092+0000", 2024-04-18T23:12:20.591 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19361344699999999 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.050246+0000", 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000153497 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.051908+0000", 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001662562 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.154762+0000", 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.102854035 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.592 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764322+0000", 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60956000700000001 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764478+0000", 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000155897 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:631 3.2s0 3:4b92d38e:::smithi14230322-19:head [setxattr _header (57) in=7b] snapc 1a=[1a,19,18,17,15,14,13,11,f,d,c] ondisk+write+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.674972314000001, 2024-04-18T23:12:20.593 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.90835886899999996, 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 631 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.594 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856506+0000", 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856507+0000", 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1000000000000001e-06 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856512+0000", 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 5.2290000000000002e-06 2024-04-18T23:12:20.595 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856534+0000", 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1356e-05 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856543+0000", 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.8060000000000001e-06 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.053052+0000", 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19650978299999999 2024-04-18T23:12:20.596 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.053209+0000", 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015716999999999999 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054047+0000", 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000837925 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.154833+0000", 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.100785849 2024-04-18T23:12:20.597 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764707+0000", 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60987347400000003 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.764865+0000", 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015817699999999999 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.598 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:632 3.2s0 3:4b92d38e:::smithi14230322-19:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e77)", 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 36.674922811000002, 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.212644217, 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 632 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:12:20.599 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856556+0000", 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.0699999999999996e-07 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.600 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856560+0000", 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.4230000000000001e-06 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856573+0000", 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.3009e-05 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:43.856581+0000", 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 8.0669999999999992e-06 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.601 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054652+0000", 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.198071467 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.054999+0000", 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00034629799999999998 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.766030+0000", 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.71103122399999996 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.602 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:44.766186+0000", 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015654300000000001 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:45.069200+0000", 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.30301327900000002 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:670 3.2s0 3:4250b374:::smithi14230322-9:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e82)", 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:48.664265+0000", 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 31.867213661000001, 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.62706280400000003, 2024-04-18T23:12:20.603 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "started", 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 670 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664265+0000", 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664265+0000", 2024-04-18T23:12:20.604 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664267+0000", 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.2239999999999998e-06 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664279+0000", 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.1919000000000001e-05 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.605 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664310+0000", 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.0855000000000002e-05 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664331+0000", 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1614999999999999e-05 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664485+0000", 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00015335899999999999 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:20.606 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:48.664494+0000", 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 9.4660000000000005e-06 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.088025+0000", 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.423531251 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.088204+0000", 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00017829 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.607 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.291327+0000", 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20312382500000001 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:741 3.2s0 3:42341748:::smithi14230322-4:head [append 0~425984] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:49.968667+0000", 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.562811382, 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91216083299999995, 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.608 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 741 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.968667+0000", 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.968667+0000", 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.609 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.968670+0000", 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8559999999999998e-06 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.969441+0000", 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00077078300000000004 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:49.969477+0000", 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 3.6038000000000003e-05 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.610 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069573+0000", 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.10009589100000001 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070763+0000", 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0011900960000000001 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071013+0000", 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025074299999999999 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.611 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.073185+0000", 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.002171849 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.270509+0000", 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19732339800000001 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.880686+0000", 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.61017750400000004 2024-04-18T23:12:20.612 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.880828+0000", 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00014167499999999999 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:742 3.2s0 3:42341748:::smithi14230322-4:head [append 0~606208] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.069661+0000", 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.461817149000002, 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.81171484500000002, 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.613 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 742 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069661+0000", 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.614 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069661+0000", 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.069664+0000", 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.8930000000000001e-06 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070530+0000", 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00086640600000000001 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070560+0000", 2024-04-18T23:12:20.615 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9538000000000001e-05 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070581+0000", 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.0483e-05 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.074800+0000", 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0042195740000000002 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.075058+0000", 2024-04-18T23:12:20.616 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00025823000000000003 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.077007+0000", 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0019490169999999999 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278274+0000", 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.201266476 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.617 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881251+0000", 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60297683199999996 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881376+0000", 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.000125396 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:743 3.2s0 3:42341748:::smithi14230322-4:head [append 0~540672] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.070618+0000", 2024-04-18T23:12:20.618 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.460860175000001, 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.81111058300000005, 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 743 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070618+0000", 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.619 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070618+0000", 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.070620+0000", 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9640000000000002e-06 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071703+0000", 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.001083044 2024-04-18T23:12:20.620 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071733+0000", 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.9699000000000001e-05 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071752+0000", 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9275000000000001e-05 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.270584+0000", 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.19883204800000001 2024-04-18T23:12:20.621 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.270766+0000", 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00018226000000000001 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.272279+0000", 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0015124030000000001 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.279437+0000", 2024-04-18T23:12:20.622 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0071585529999999998 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881609+0000", 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.60217155700000002 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.881729+0000", 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011977999999999999 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.623 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:744 3.2s0 3:42341748:::smithi14230322-4:head [append 0~409600] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.071789+0000", 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.459689761, 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91037796400000004, 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 744 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.624 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071789+0000", 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071789+0000", 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.071791+0000", 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 2.1459999999999999e-06 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.625 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072442+0000", 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00065123700000000004 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072461+0000", 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9485999999999999e-05 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072479+0000", 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.7892000000000001e-05 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.626 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.273674+0000", 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.20119500400000001 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.273959+0000", 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00028440799999999999 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.277315+0000", 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.003356412 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.627 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.281415+0000", 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0041003360000000004 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982028+0000", 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.70061242700000004 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982167+0000", 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00013861599999999999 2024-04-18T23:12:20.628 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "description": "osd_op(client.4568.0:745 3.2s0 3:42341748:::smithi14230322-4:head [setxattr _header (56) in=7b] snapc 1b=[1b,19,17,15,13,f,d] ondisk+write+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "initiated_at": "2024-04-18T23:11:50.072506+0000", 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "age": 30.458972178, 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.91007206600000001, 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "continuous": false, 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "type_data": { 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "client_info": { 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "client": "client.4568", 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: "tid": 745 2024-04-18T23:12:20.629 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: "events": [ 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "initiated", 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072506+0000", 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "header_read", 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072506+0000", 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "throttled", 2024-04-18T23:12:20.630 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072508+0000", 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.4470000000000001e-06 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "all_read", 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072523+0000", 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5756e-05 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "dispatched", 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072543+0000", 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.9712e-05 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "queued_for_pg", 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.072559+0000", 2024-04-18T23:12:20.631 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 1.5758999999999999e-05 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "reached_pg", 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278314+0000", 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.205755145 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "started", 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278426+0000", 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00011250499999999999 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_started", 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.278921+0000", 2024-04-18T23:12:20.632 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00049428099999999995 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "sub_op_committed", 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.281726+0000", 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.0028056639999999998 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "commit_sent", 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982417+0000", 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.70069052600000004 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: }, 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: { 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: "event": "done", 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: "time": "2024-04-18T23:11:50.982578+0000", 2024-04-18T23:12:20.633 INFO:teuthology.orchestra.run.smithi120.stdout: "duration": 0.00016127099999999999 2024-04-18T23:12:20.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.634 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.634 INFO:teuthology.orchestra.run.smithi120.stdout: } 2024-04-18T23:12:20.634 INFO:teuthology.orchestra.run.smithi120.stdout: ] 2024-04-18T23:12:20.634 INFO:teuthology.orchestra.run.smithi120.stdout:} 2024-04-18T23:12:20.634 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T23:12:20.636 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:20.636+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:20.637 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-18T23:12:20.671 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.5 to 1 (10000) 2024-04-18T23:12:20.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-18T23:12:20.736 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:20.736+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:20.770 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:20.771 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:20.771 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:12:20.771 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:20.785 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-18T23:12:20.837 INFO:tasks.ceph.osd.1.smithi120.stderr:2024-04-18T23:12:20.837+0000 7f2fada09640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 25054) UID: 0 2024-04-18T23:12:20.938 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:20.936+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:20.950 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:20.950 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:20.950 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:12:20.950 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:12:20.951 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:20.964 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-18T23:12:21.038 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:21.039+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:21.133 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:133 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'91, mlcod=28'85)", 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 91.854444810000004, 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20553276400000001, 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.134 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277422+0000", 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277424+0000", 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.21e-06 2024-04-18T23:12:21.135 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277436+0000", 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1693e-05 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.277464+0000", 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8608000000000001e-05 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377595+0000", 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100130684 2024-04-18T23:12:21.136 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377730+0000", 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013455499999999999 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.377781+0000", 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1406999999999998e-05 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482870+0000", 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.105089222 2024-04-18T23:12:21.137 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:49.482955+0000", 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4384999999999999e-05 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:155 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'109, mlcod=28'105)", 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.996906699999997, 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20593233399999999, 2024-04-18T23:12:21.138 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134960+0000", 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.139 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.134962+0000", 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8560000000000001e-06 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135074+0000", 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000112357 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135099+0000", 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4705000000000001e-05 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.140 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135115+0000", 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5855999999999999e-05 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135494+0000", 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00037909800000000002 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.135551+0000", 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7176999999999998e-05 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.141 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340852+0000", 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20530157900000001 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.340892+0000", 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9706000000000002e-05 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_repop(client.4260.0:157 1.0 e28/22 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 28'111, mlcod=28'105)", 2024-04-18T23:12:21.142 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 90.995385381000006, 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20611499799999999, 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136481+0000", 2024-04-18T23:12:21.143 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136483+0000", 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.576e-06 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136499+0000", 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6529e-05 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136520+0000", 2024-04-18T23:12:21.144 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0571999999999999e-05 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.136532+0000", 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.223e-05 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237736+0000", 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10120357200000001 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.237776+0000", 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0397e-05 2024-04-18T23:12:21.145 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342561+0000", 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104784615 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:50.342596+0000", 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5506999999999997e-05 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.146 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:201 3.es0 3:78aee268:::smithi14230322-13:2 [read 0~212992,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.570110962000001, 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.41057925299999998, 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 201 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.147 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561756+0000", 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561757+0000", 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.47e-07 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.148 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561762+0000", 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.04e-06 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561777+0000", 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4962e-05 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.561785+0000", 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4500000000000004e-06 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.149 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562118+0000", 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000332882 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.562139+0000", 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1101999999999999e-05 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.769814+0000", 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.207675318 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.150 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770164+0000", 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00034956199999999998 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.972335+0000", 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20217098999999999 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(osd.4.21:1 3.es0 3:7a4badf6:::smithi14230322-17:head [assert-version v4,copy-get max 8388608] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e36)", 2024-04-18T23:12:21.151 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 80.370334129, 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31203379199999998, 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761532+0000", 2024-04-18T23:12:21.152 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761535+0000", 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7999999999999999e-06 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761547+0000", 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1279e-05 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761568+0000", 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1549e-05 2024-04-18T23:12:21.153 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.761588+0000", 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0426000000000001e-05 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.770853+0000", 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0092644660000000007 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:00.771105+0000", 2024-04-18T23:12:21.154 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025160800000000001 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:01.073566+0000", 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30246166400000002 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:333 3.es0 3:799dcc49:::smithi14230322-39:head [create,setxattr Yjk]k``VRNVkfm]Rnjieeerecjk`Wm (15648) in=30b,setxattr QgpqijWTrohYWOsYikii[YdsSee`QrtrZ]nSPTUt (1225) in=40b,setxattr l]iRdbVhahjiOeVf`Yc\\U_OYjY]nr[js\\NggsVVY (1015) in=40b,setxattr b^W[ZgXWiRmfON^`rihabiO]lNkro`P (15648) in=31b,setxattr VReWeRiYgNSZU\\d\\YOrNS`]ogtdOjVijWVeY`fhd (1051) in=40b,setxattr SYsfepqPdsc`cUg\\WaTfgrcUYhUbTiaQY_`rlt (15648) in=38b,setxattr P[`jheVP[fjpnXqdjPhddi_oWUf\\dWWbSlodc`eS (1086) in=40b,setxattr X[^VkT`QVNea_jfmlVkdafY]hrorgVhl]fqtiVie (1226) in=40b,setxattr Xs\\WjfX^_Ss_RPr_tT^a]fnSSlYrcPTbmSaksi_c (19437) in=40b,setxattr QUn_^p]__nRhn\\srj]jla^iQSPOpgQcVcQZU^ool (1823) in=40b,setxattr f\\^WXdjheR\\aQSYk]Un_^hkrZTSVdhiZqgcin[Sa (1155) in=40b,setxattr bkb]mNeZddhQrcsORPSgqbWecOTtcPYZaRQOg`nY (1614) in=40b,setxattr _T_atc\\\\Rml^aOPkNsbsb^VatOWSs\\URSjVqfe\\Z (1226) in=40b,setxattr obYqlk]hgNOQgXa^ZhalmPcXl\\lkOdNNiYlOfdhq (1612) in=40b,setxattr VsbnoacTrXVpb]PV]kVPYbQP_qefUdfOgk`qUaNa (1999) in=40b,setxattr ZqTlaiiPajkNT^ce\\W\\oklalkiT`\\\\nhjcgpZrUa (3297) in=40b,setxattr QfUmVl\\][PRg\\ajWQS`]Oe^Zlig]b]Rs_o]NsY[i (244) in=40b,setxattr R_Ucr`a\\j]Ri_YSp[gamgRSRd`]UYnPRspctZ`R] (4068) in=40b,setxattr aj`m[hpi__Rrg_Z[m^rVOVrVgk^cS_]piWV]lkRZ (947) in=40b,setxattr ^nWmd_TeciTqUOnWXn_jXch]heNac[WdcqdPbiT^ (1647) in=40b] snapc 8=[7,4] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.504510766999999, 2024-04-18T23:12:21.155 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30962595399999998, 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 333 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.156 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627356+0000", 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627359+0000", 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3400000000000002e-06 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.157 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627676+0000", 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000317144 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627710+0000", 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3855e-05 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627753+0000", 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3341999999999999e-05 2024-04-18T23:12:21.158 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627926+0000", 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017248799999999999 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.627946+0000", 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9868999999999999e-05 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729527+0000", 2024-04-18T23:12:21.159 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101581587 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.729747+0000", 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021955299999999999 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.733259+0000", 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0035115820000000001 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739147+0000", 2024-04-18T23:12:21.160 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0058886169999999996 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936726+0000", 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.197578422 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.936982+0000", 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025615499999999999 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:344 3.es0 3:7a4badf6:::smithi14230322-17:head [delete] snapc 8=[7,4] ondisk+write+peerstat_old+known_if_redirected+supports_pool_eio e47)", 2024-04-18T23:12:21.161 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 68.490452210000001, 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.295980827, 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 344 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:12:21.162 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641414+0000", 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641417+0000", 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2630000000000002e-06 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641427+0000", 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0674e-05 2024-04-18T23:12:21.163 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641460+0000", 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2768999999999997e-05 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641482+0000", 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1959000000000001e-05 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641642+0000", 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016009100000000001 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.164 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.641653+0000", 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.128e-05 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735170+0000", 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.093517063999999997 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.735466+0000", 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029521400000000002 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.165 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.736750+0000", 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012846540000000001 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.739533+0000", 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0027831589999999999 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937242+0000", 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19770827199999999 2024-04-18T23:12:21.166 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:12.937395+0000", 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000153428 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 56/49 ECSubWrite(tid=94, reqid=client.4568.0:421, at_version=56'16, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.488481790999998, 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.27351094799999998, 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.167 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643385+0000", 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.168 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643386+0000", 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4300000000000002e-07 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643802+0000", 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000416136 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643934+0000", 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013215300000000001 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.169 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.643959+0000", 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4973e-05 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.812936+0000", 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.168977035 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.812952+0000", 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6419e-05 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.170 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916818+0000", 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10386601099999999 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916896+0000", 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7378000000000004e-05 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s1 56/49 ECSubWrite(tid=95, reqid=client.4568.0:422, at_version=56'17, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 59.482866541, 2024-04-18T23:12:21.171 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.26796196300000003, 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649000+0000", 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.172 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649002+0000", 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8199999999999999e-06 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649359+0000", 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035705699999999998 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.649390+0000", 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.146e-05 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.173 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749486+0000", 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100095925 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.814696+0000", 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.065209634000000002 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.814713+0000", 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7082999999999999e-05 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.174 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916904+0000", 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10219086500000001 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.916962+0000", 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.8118999999999997e-05 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:467 3.es0 3:799dcc49:::smithi14230322-39:head [read 0~638976,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:12:21.175 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.578225080999999, 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.39041841700000002, 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 467 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.176 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553641+0000", 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553642+0000", 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3000000000000001e-07 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553645+0000", 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.813e-06 2024-04-18T23:12:21.177 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553650+0000", 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6840000000000001e-06 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.553655+0000", 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2080000000000002e-06 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654393+0000", 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100738487 2024-04-18T23:12:21.178 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.654401+0000", 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9340000000000006e-06 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.657013+0000", 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0026115800000000001 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.657158+0000", 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014565499999999999 2024-04-18T23:12:21.179 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.944060+0000", 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28690162600000002 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=42, last_complete=61'29, committed=1, applied=1))", 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 54.372412167, 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28503450299999999, 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.180 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759454+0000", 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.181 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759455+0000", 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7100000000000001e-07 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759459+0000", 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1760000000000003e-06 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.759471+0000", 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1523e-05 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.182 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.859553+0000", 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10008174 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.044450+0000", 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.18489760999999999 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.044489+0000", 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8683000000000002e-05 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.183 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=44, last_complete=61'31, committed=1, applied=1))", 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.772663786000003, 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29027006799999999, 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:12:21.184 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359203+0000", 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359204+0000", 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.282e-06 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.185 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359210+0000", 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6579999999999999e-06 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359227+0000", 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7283999999999999e-05 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.359257+0000", 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9643000000000001e-05 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.186 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649393+0000", 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.29013631200000001 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649473+0000", 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9888999999999994e-05 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWriteReply(3.es0 61/31 ECSubWriteReply(tid=45, last_complete=61'33, committed=1, applied=1))", 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:12:21.187 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 53.771532546000003, 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28925603500000002, 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360334+0000", 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.188 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360335+0000", 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.8999999999999995e-07 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360340+0000", 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4160000000000003e-06 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360354+0000", 2024-04-18T23:12:21.189 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3709999999999999e-05 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.360362+0000", 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6120000000000001e-06 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649559+0000", 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.28919758400000001 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.190 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.649590+0000", 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0822999999999999e-05 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:497 3.es0 3:799dcc49:::smithi14230322-39:head [rmxattr QgpqijWTrohYWOsYikii[YdsSee`QrtrZ]nSPTUt in=40b,rmxattr R_Ucr`a\\j]Ri_YSp[gamgRSRd`]UYnPRspctZ`R] in=40b,rmxattr VReWeRiYgNSZU\\d\\YOrNS`]ogtdOjVijWVeY`fhd in=40b,rmxattr X[^VkT`QVNea_jfmlVkdafY]hrorgVhl]fqtiVie in=40b,rmxattr _T_atc\\\\Rml^aOPkNsbsb^VatOWSs\\URSjVqfe\\Z in=40b,rmxattr bkb]mNeZddhQrcsORPSgqbWecOTtcPYZaRQOg`nY in=40b] snapc 15=[15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e66)", 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 49.601229607, 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.21045935599999999, 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.191 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 497 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530637+0000", 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.192 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530641+0000", 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7869999999999999e-06 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530672+0000", 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1642e-05 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530707+0000", 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4566000000000001e-05 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.193 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530726+0000", 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9130000000000001e-05 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530864+0000", 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000138307 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "op must wait for map", 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.530878+0000", 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.379e-05 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.194 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.534931+0000", 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.004053035 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.535122+0000", 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000190612 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.536668+0000", 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0015465800000000001 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.640407+0000", 2024-04-18T23:12:21.195 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103738334 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.740945+0000", 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100538086 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:31.741096+0000", 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00015148700000000001 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.196 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:551 3.es0 3:78aee268:::smithi14230322-13:head [append 0~425984] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e69)", 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.563216+0000", 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.568650394000002, 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31486470599999999, 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 551 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.197 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.563216+0000", 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.563216+0000", 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.563219+0000", 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.92e-06 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.198 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564232+0000", 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010124229999999999 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564266+0000", 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4282000000000001e-05 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564286+0000", 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0134000000000001e-05 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.199 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564452+0000", 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000165876 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.564647+0000", 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00019565699999999999 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.566327+0000", 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001679313 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.200 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.568134+0000", 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0018070339999999999 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.877907+0000", 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30977282 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878081+0000", 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017424699999999999 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.201 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:552 3.es0 3:78aee268:::smithi14230322-13:head [append 0~786432] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.570285+0000", 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.561581318000002, 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30865453999999998, 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 552 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.202 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.570285+0000", 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.570285+0000", 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.570295+0000", 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3340000000000008e-06 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.203 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.571762+0000", 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001467007 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.571793+0000", 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1303000000000002e-05 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.571808+0000", 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4853e-05 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.204 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.670635+0000", 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098827154 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.670823+0000", 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00018851999999999999 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.672550+0000", 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0017261030000000001 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.777055+0000", 2024-04-18T23:12:21.205 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10450546500000001 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878818+0000", 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10176346999999999 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878940+0000", 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00012133099999999999 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.206 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:553 3.es0 3:78aee268:::smithi14230322-13:head [append 0~376832] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.573318+0000", 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.558548088000002, 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.40648862800000002, 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 553 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.207 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.573318+0000", 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.573318+0000", 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.573320+0000", 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5239999999999999e-06 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.208 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574342+0000", 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0010218429999999999 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574366+0000", 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3689999999999998e-05 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574378+0000", 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2386e-05 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.209 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.774530+0000", 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20015175199999999 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.774703+0000", 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017317299999999999 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.776150+0000", 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0014476020000000001 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.210 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.878103+0000", 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101952266 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.979668+0000", 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101564918 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.979807+0000", 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000139474 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.211 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:554 3.es0 3:78aee268:::smithi14230322-13:head [setxattr _header (57) in=7b] snapc 16=[16,15,14,13,12,11,f,d,c,9] ondisk+write+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.574400+0000", 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.557466652000002, 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.50849109599999998, 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 554 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.212 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574400+0000", 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574400+0000", 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574401+0000", 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.203e-06 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.213 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574407+0000", 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.207e-06 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574418+0000", 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0257e-05 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574425+0000", 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7319999999999998e-06 2024-04-18T23:12:21.214 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.879281+0000", 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30485609800000002 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.879453+0000", 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017140699999999999 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.880371+0000", 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00091843199999999995 2024-04-18T23:12:21.215 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.881707+0000", 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001335943 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "commit_sent", 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.082730+0000", 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201023233 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.082891+0000", 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00016058399999999999 2024-04-18T23:12:21.216 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:555 3.es0 3:78aee268:::smithi14230322-13:head [read 0~1] snapc 0=[] ondisk+read+rwordered+known_if_redirected+supports_pool_eio e70)", 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:36.574436+0000", 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.557430083, 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.81095218999999996, 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.217 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 555 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574436+0000", 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574436+0000", 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.218 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574437+0000", 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7700000000000004e-07 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574441+0000", 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4599999999999999e-06 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.574451+0000", 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.012e-05 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.219 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.674559+0000", 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100107769 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.880898+0000", 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20633925 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting for rw locks", 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:36.881048+0000", 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014995699999999999 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.220 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.083603+0000", 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202554858 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.083750+0000", 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014693 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.385389+0000", 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30163906899999998 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.221 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "osd_op(client.4568.0:683 3.es0 3:78aee268:::smithi14230322-13:head [read 0~4669440,checksum] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e83)", 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:49.796228+0000", 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.335638328000002, 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.31501509300000002, 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "started", 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "client_info": { 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "client": "client.4568", 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "client_addr": "172.21.15.142:0/1532284881", 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: "tid": 683 2024-04-18T23:12:21.222 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796228+0000", 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796228+0000", 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796229+0000", 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7899999999999998e-07 2024-04-18T23:12:21.223 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796234+0000", 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5180000000000004e-06 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796246+0000", 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1525000000000001e-05 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796253+0000", 2024-04-18T23:12:21.224 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9870000000000002e-06 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796466+0000", 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00021326400000000001 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "waiting_for_map not empty", 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.796477+0000", 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.06e-05 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.801736+0000", 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0052596689999999998 2024-04-18T23:12:21.225 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "started", 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:49.801835+0000", 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.8825000000000003e-05 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.111243+0000", 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30940812600000001 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.226 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:21.228 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-18T23:12:21.228 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:21.228+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:21.329 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:21.328+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:21.398 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:21.398 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:21.398 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:12:21.398 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:21.412 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-18T23:12:21.429 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:21.426+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:21.530 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:21.526+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:21.577 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:21.577 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:21.577 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:12:21.577 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:12:21.577 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:21.591 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-18T23:12:21.631 INFO:tasks.ceph.osd.11.smithi142.stderr:2024-04-18T23:12:21.631+0000 7fb5b111a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 11 (PID: 25342) UID: 0 2024-04-18T23:12:21.637 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:12:21.679 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-04-18T23:12:21.679 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.6 to 1 (10000) 2024-04-18T23:12:21.705 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=158, reqid=client.4568.0:11, at_version=33'2, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 85.084930091000004, 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203058724, 2024-04-18T23:12:21.714 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627729+0000", 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.715 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627730+0000", 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.606e-06 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627841+0000", 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000110692 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627889+0000", 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8211000000000003e-05 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.716 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.627919+0000", 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9961999999999999e-05 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629700+0000", 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001780963 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.629730+0000", 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0082000000000001e-05 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.717 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830736+0000", 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201005607 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830787+0000", 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.1601000000000003e-05 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=159, reqid=client.4568.0:25, at_version=33'3, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:21.718 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 85.084656277999997, 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20284285199999999, 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628003+0000", 2024-04-18T23:12:21.719 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628004+0000", 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0270000000000001e-06 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628137+0000", 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013336000000000001 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.720 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628197+0000", 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.0504000000000002e-05 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628221+0000", 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3037999999999999e-05 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728171+0000", 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099950707999999999 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.721 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728190+0000", 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8709e-05 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830795+0000", 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102605138 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830845+0000", 2024-04-18T23:12:21.722 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0368000000000003e-05 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 33/31 ECSubWrite(tid=160, reqid=client.4568.0:26, at_version=33'4, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 85.084401557000007, 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20264686400000001, 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:12:21.723 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628257+0000", 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628258+0000", 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628292+0000", 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.4054999999999998e-05 2024-04-18T23:12:21.724 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628317+0000", 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5035999999999999e-05 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628338+0000", 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1058999999999999e-05 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728828+0000", 2024-04-18T23:12:21.725 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100489521 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.728846+0000", 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8419000000000001e-05 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830850+0000", 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10200397 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.726 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830904+0000", 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3844000000000002e-05 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.2s3 34/31 ECSubWrite(tid=161, reqid=client.4568.0:31, at_version=34'5, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 85.084267513, 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202594633, 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.727 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628391+0000", 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628392+0000", 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.6000000000000004e-07 2024-04-18T23:12:21.728 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628541+0000", 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149386 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628572+0000", 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0975000000000001e-05 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.628594+0000", 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1608999999999999e-05 2024-04-18T23:12:21.729 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829299+0000", 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20070527499999999 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.829314+0000", 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4718999999999999e-05 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830915+0000", 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001601412 2024-04-18T23:12:21.730 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:10:56.830986+0000", 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.0697000000000004e-05 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 56/31 ECSubWrite(tid=164, reqid=client.4568.0:431, at_version=56'73, trim_to=0'0, roll_forward_to=55'71))", 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 60.167146971999998, 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20358216400000001, 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.731 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.732 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545512+0000", 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7300000000000002e-07 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545582+0000", 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9449000000000005e-05 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545607+0000", 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5741999999999998e-05 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.545625+0000", 2024-04-18T23:12:21.733 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7503000000000002e-05 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.645851+0000", 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100225733 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.645870+0000", 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8972999999999999e-05 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.734 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749022+0000", 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103152813 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:21.749094+0000", 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1577999999999998e-05 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.5s6 70/68 ECSubWrite(tid=81, reqid=client.4568.0:550, at_version=70'42, trim_to=0'0, roll_forward_to=69'41))", 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:37.152485+0000", 2024-04-18T23:12:21.735 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 44.560174216, 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201685279, 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152485+0000", 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152485+0000", 2024-04-18T23:12:21.736 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152486+0000", 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0999999999999997e-07 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152536+0000", 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.0312000000000002e-05 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.737 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152560+0000", 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4046e-05 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.152580+0000", 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9792999999999998e-05 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.352861+0000", 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20028146199999999 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.738 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.352877+0000", 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5420000000000001e-05 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.354095+0000", 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0012185469999999999 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:37.354170+0000", 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4789000000000005e-05 2024-04-18T23:12:21.739 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 71/70 ECSubWrite(tid=35, reqid=osd.5.0:35, at_version=71'26, trim_to=0'0, roll_forward_to=71'22))", 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.092364+0000", 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.620294803, 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20238825899999999, 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.740 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092364+0000", 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092364+0000", 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092365+0000", 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.373e-06 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.741 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092404+0000", 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8241999999999997e-05 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092428+0000", 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4729999999999999e-05 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.092443+0000", 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4883e-05 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.742 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.292777+0000", 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.200333228 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.292791+0000", 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4059e-05 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.294674+0000", 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.001883205 2024-04-18T23:12:21.743 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.294752+0000", 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8539000000000001e-05 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.8s6 72/70 ECSubWrite(tid=40, reqid=osd.5.0:39, at_version=0'0, trim_to=0'0, roll_forward_to=72'30))", 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:39.022976+0000", 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.689682931, 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20188773400000001, 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.744 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.022976+0000", 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.022976+0000", 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.745 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.022978+0000", 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6169999999999999e-06 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.023000+0000", 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2660999999999999e-05 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.023038+0000", 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8104e-05 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.023054+0000", 2024-04-18T23:12:21.746 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6062e-05 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.123257+0000", 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10020230199999999 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.123276+0000", 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8842999999999999e-05 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.224763+0000", 2024-04-18T23:12:21.747 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101487857 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.224864+0000", 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000100288 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.0s5 72/71 ECSubWrite(tid=149, reqid=osd.14.0:149, at_version=72'26, trim_to=0'0, roll_forward_to=0'0))", 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:39.024690+0000", 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 42.687968783999999, 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20278465600000001, 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.748 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024690+0000", 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024690+0000", 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.749 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024692+0000", 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.813e-06 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024756+0000", 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.4202000000000006e-05 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024796+0000", 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9762999999999999e-05 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.750 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.024821+0000", 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4688000000000001e-05 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.223945+0000", 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19912484499999999 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.223959+0000", 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3985000000000001e-05 2024-04-18T23:12:21.751 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.227409+0000", 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.003449428 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.227475+0000", 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5932000000000005e-05 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.2s3 77/67 ECSubRead(tid=217, to_read={3:42c34e84:::smithi14230322-11:head=0,458752,0}, subchunks={3:42c34e84:::smithi14230322-11:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:21.752 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:43.843070+0000", 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 37.869588649999997, 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30067776099999999, 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843070+0000", 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843070+0000", 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.753 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843072+0000", 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6169999999999999e-06 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843087+0000", 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5522000000000001e-05 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.754 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843118+0000", 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0859999999999999e-05 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:43.843143+0000", 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4766e-05 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.143457+0000", 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30031421600000002 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.755 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.143748+0000", 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00029077999999999998 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 78/31 ECSubWrite(tid=264, reqid=osd.4.0:260, at_version=0'0, trim_to=0'0, roll_forward_to=78'134))", 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.818594+0000", 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.894064757000002, 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20360535399999999, 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.756 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818594+0000", 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818594+0000", 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818596+0000", 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8870000000000001e-06 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.757 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818629+0000", 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3408000000000003e-05 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818665+0000", 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5160000000000002e-05 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818689+0000", 2024-04-18T23:12:21.758 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4655e-05 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818836+0000", 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00014703000000000001 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.818853+0000", 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6636000000000001e-05 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022114+0000", 2024-04-18T23:12:21.759 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203261577 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022199+0000", 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.5000999999999995e-05 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 78/31 ECSubWrite(tid=265, reqid=osd.4.0:265, at_version=78'136, trim_to=0'0, roll_forward_to=78'134))", 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:44.819697+0000", 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 36.892961618999998, 2024-04-18T23:12:21.760 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202551706, 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819697+0000", 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819697+0000", 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.761 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819698+0000", 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.24e-06 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819756+0000", 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7926000000000001e-05 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819792+0000", 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5364000000000002e-05 2024-04-18T23:12:21.762 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.819815+0000", 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2861999999999999e-05 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.920043+0000", 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100228418 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:44.920060+0000", 2024-04-18T23:12:21.763 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6769000000000002e-05 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022212+0000", 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10215207 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:45.022249+0000", 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7057000000000001e-05 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.764 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 83/31 ECSubWrite(tid=326, reqid=client.4568.0:730, at_version=83'170, trim_to=0'0, roll_forward_to=82'166))", 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.355423+0000", 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.357236067999999, 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202790049, 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355423+0000", 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.765 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355423+0000", 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355423+0000", 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.4000000000000002e-07 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355502+0000", 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.9090000000000003e-05 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355511+0000", 2024-04-18T23:12:21.766 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1779999999999992e-06 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355526+0000", 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4688999999999999e-05 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.455687+0000", 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10016045 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.455697+0000", 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0730999999999999e-05 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.767 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558181+0000", 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102483502 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558213+0000", 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1968999999999998e-05 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 83/31 ECSubWrite(tid=327, reqid=client.4568.0:731, at_version=83'171, trim_to=0'0, roll_forward_to=82'166))", 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.355547+0000", 2024-04-18T23:12:21.768 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.357112274999999, 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20268904900000001, 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355547+0000", 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355547+0000", 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.769 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355547+0000", 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6399999999999998e-07 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355623+0000", 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.6070999999999997e-05 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355634+0000", 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1194e-05 2024-04-18T23:12:21.770 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355643+0000", 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.6920000000000008e-06 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.456260+0000", 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100616868 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.771 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.456290+0000", 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9867999999999999e-05 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558218+0000", 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10192849900000001 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558236+0000", 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7493e-05 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.772 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 83/31 ECSubWrite(tid=328, reqid=client.4568.0:732, at_version=83'172, trim_to=0'0, roll_forward_to=83'168))", 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.355659+0000", 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.357000097, 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20262286099999999, 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355659+0000", 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.773 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355659+0000", 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355659+0000", 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.7399999999999999e-07 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355705+0000", 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.6103000000000002e-05 2024-04-18T23:12:21.774 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355717+0000", 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1939e-05 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355726+0000", 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1710000000000008e-06 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.557073+0000", 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20134684799999999 2024-04-18T23:12:21.775 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.557091+0000", 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7858000000000001e-05 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558262+0000", 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0011712820000000001 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558282+0000", 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9286000000000001e-05 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.776 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 83/31 ECSubWrite(tid=329, reqid=client.4568.0:733, at_version=83'173, trim_to=0'0, roll_forward_to=83'168))", 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.355740+0000", 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.356919344000001, 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202871142, 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355740+0000", 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.777 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355740+0000", 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355740+0000", 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.53e-07 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355752+0000", 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2571999999999999e-05 2024-04-18T23:12:21.778 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355762+0000", 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5529999999999998e-06 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.355771+0000", 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0329999999999997e-06 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.557972+0000", 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20220071000000001 2024-04-18T23:12:21.779 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.557981+0000", 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.4590000000000004e-06 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558569+0000", 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00058762599999999995 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.558611+0000", 2024-04-18T23:12:21.780 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1835999999999997e-05 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.bs4 94/31 ECSubWrite(tid=406, reqid=client.4568.0:826, at_version=94'168, trim_to=0'0, roll_forward_to=92'163))", 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:00.560036+0000", 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 21.152622578999999, 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20184205999999999, 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.781 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560036+0000", 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560036+0000", 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560038+0000", 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.624e-06 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.782 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560395+0000", 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00035757699999999998 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560419+0000", 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3462e-05 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.560433+0000", 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.393e-05 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.783 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.759783+0000", 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19934999 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.759799+0000", 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6370000000000001e-05 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.761815+0000", 2024-04-18T23:12:21.784 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0020154130000000002 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:00.761878+0000", 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.3694000000000004e-05 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 99/31 ECSubWrite(tid=462, reqid=osd.4.0:460, at_version=0'0, trim_to=0'0, roll_forward_to=99'242))", 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.763334+0000", 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.949324732000001, 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20324613, 2024-04-18T23:12:21.785 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763334+0000", 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763334+0000", 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.786 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763336+0000", 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.9360000000000002e-06 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763358+0000", 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2229e-05 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763383+0000", 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4528999999999998e-05 2024-04-18T23:12:21.787 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763401+0000", 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7753000000000001e-05 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.863697+0000", 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100296253 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.863728+0000", 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1158999999999998e-05 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.788 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.966495+0000", 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10276741 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.966580+0000", 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.4860999999999999e-05 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 99/31 ECSubWrite(tid=463, reqid=osd.4.0:463, at_version=99'244, trim_to=0'0, roll_forward_to=99'242))", 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.763441+0000", 2024-04-18T23:12:21.789 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.949218139999999, 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20321472800000001, 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763441+0000", 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763441+0000", 2024-04-18T23:12:21.790 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763441+0000", 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7000000000000005e-07 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763477+0000", 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5444999999999997e-05 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763490+0000", 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3467e-05 2024-04-18T23:12:21.791 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763501+0000", 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0458999999999999e-05 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.864470+0000", 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100968844 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.864502+0000", 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2938999999999998e-05 2024-04-18T23:12:21.792 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.966593+0000", 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102091027 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.966655+0000", 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.1977000000000006e-05 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.793 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.7s1 99/31 ECSubWrite(tid=464, reqid=osd.4.0:464, at_version=99'246, trim_to=0'0, roll_forward_to=99'242))", 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.763522+0000", 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 14.949136506, 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20376124000000001, 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763522+0000", 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.794 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763522+0000", 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763523+0000", 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.8999999999999997e-07 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763552+0000", 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.9371999999999999e-05 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.795 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763564+0000", 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1518999999999999e-05 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.763573+0000", 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1570000000000006e-06 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.965500+0000", 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20192695399999999 2024-04-18T23:12:21.796 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.965531+0000", 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0673000000000003e-05 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.967208+0000", 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0016772379999999999 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.967284+0000", 2024-04-18T23:12:21.797 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.5836999999999999e-05 2024-04-18T23:12:21.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.798 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.798 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:21.798 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:21.798 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:21.800 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_ops_in_flight 2024-04-18T23:12:21.801 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:21.797+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:21.901 INFO:tasks.ceph.osd.14.smithi136.stderr:2024-04-18T23:12:21.900+0000 7f14a03e2640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 14 (PID: 25066) UID: 0 2024-04-18T23:12:21.929 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:21.929 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:21.929 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:12:21.929 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:21.943 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_blocked_ops 2024-04-18T23:12:21.954 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:12:21.955 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:12:21.972 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-04-18T23:12:21.677066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":173163040,"num_objects":144,"num_object_clones":92,"num_object_copies":998,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":144,"num_whiteouts":7,"num_read":616,"num_read_kb":187492,"num_write":1101,"num_write_kb":190180,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1669,"ondisk_log_size":1669,"up":130,"acting":130,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":16,"kb":1509949440,"kb_used":823496,"kb_used_data":317484,"kb_used_omap":35,"kb_used_meta":505756,"kb_avail":1509125944,"statfs":{"total":1546188226560,"available":1545344966656,"internally_reserved":0,"allocated":325103616,"data_stored":474072730,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":517894785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.030920"},"pg_stats":[{"pgid":"3.f","version":"103'23","reported_seq":370,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.067029+0000","last_change":"2024-04-18T23:12:09.804055+0000","last_active":"2024-04-18T23:12:20.067029+0000","last_peered":"2024-04-18T23:12:20.067029+0000","last_clean":"2024-04-18T23:12:20.067029+0000","last_became_active":"2024-04-18T23:10:54.696778+0000","last_became_peered":"2024-04-18T23:10:54.696778+0000","last_unstale":"2024-04-18T23:12:20.067029+0000","last_undegraded":"2024-04-18T23:12:20.067029+0000","last_fullsized":"2024-04-18T23:12:20.067029+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.20648823399999999,"stat_sum":{"num_bytes":4521984,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":20,"num_read_kb":19110,"num_write":7,"num_write_kb":4416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,14,13,8,10,0],"acting":[11,2,14,13,8,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"103'96","reported_seq":441,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.075235+0000","last_change":"2024-04-18T23:12:10.305502+0000","last_active":"2024-04-18T23:12:20.075235+0000","last_peered":"2024-04-18T23:12:20.075235+0000","last_clean":"2024-04-18T23:12:20.075235+0000","last_became_active":"2024-04-18T23:10:54.918517+0000","last_became_peered":"2024-04-18T23:10:54.918517+0000","last_unstale":"2024-04-18T23:12:20.075235+0000","last_undegraded":"2024-04-18T23:12:20.075235+0000","last_fullsized":"2024-04-18T23:12:20.075235+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'84","last_scrub_stamp":"2024-04-18T23:12:01.429717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.429717+0000","objects_scrubbed":6,"log_size":96,"log_dups_size":0,"ondisk_log_size":96,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:07.931630+0000","scrub_duration":815,"objects_trimmed":0,"snaptrim_duration":0.60798597700000001,"stat_sum":{"num_bytes":14565376,"num_objects":8,"num_object_clones":5,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":16278,"num_write":60,"num_write_kb":15408,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,1,11,8,6,12],"acting":[2,9,1,11,8,6,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"103'172","reported_seq":523,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-18T23:12:16.746436+0000","last_change":"2024-04-18T23:12:10.003290+0000","last_active":"2024-04-18T23:12:16.746436+0000","last_peered":"2024-04-18T23:12:16.746436+0000","last_clean":"2024-04-18T23:12:16.746436+0000","last_became_active":"2024-04-18T23:10:54.712663+0000","last_became_peered":"2024-04-18T23:10:54.712663+0000","last_unstale":"2024-04-18T23:12:16.746436+0000","last_undegraded":"2024-04-18T23:12:16.746436+0000","last_fullsized":"2024-04-18T23:12:16.746436+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'148","last_scrub_stamp":"2024-04-18T23:12:03.759433+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:03.759433+0000","objects_scrubbed":13,"log_size":172,"log_dups_size":0,"ondisk_log_size":172,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:15.272416+0000","scrub_duration":319,"objects_trimmed":1,"snaptrim_duration":0.29862492400000001,"stat_sum":{"num_bytes":18448384,"num_objects":15,"num_object_clones":10,"num_object_copies":105,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":1,"num_read":58,"num_read_kb":12309,"num_write":120,"num_write_kb":21648,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,14,10,0,8,4],"acting":[7,3,14,10,0,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"}]},{"pgid":"3.c","version":"103'77","reported_seq":427,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.066866+0000","last_change":"2024-04-18T23:12:11.003242+0000","last_active":"2024-04-18T23:12:20.066866+0000","last_peered":"2024-04-18T23:12:20.066866+0000","last_clean":"2024-04-18T23:12:20.066866+0000","last_became_active":"2024-04-18T23:10:54.717053+0000","last_became_peered":"2024-04-18T23:10:54.717053+0000","last_unstale":"2024-04-18T23:12:20.066866+0000","last_undegraded":"2024-04-18T23:12:20.066866+0000","last_fullsized":"2024-04-18T23:12:20.066866+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'77","last_scrub_stamp":"2024-04-18T23:12:11.002641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:11.002641+0000","objects_scrubbed":8,"log_size":77,"log_dups_size":0,"ondisk_log_size":77,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:20.747219+0000","scrub_duration":315,"objects_trimmed":1,"snaptrim_duration":1.028340676,"stat_sum":{"num_bytes":17563648,"num_objects":8,"num_object_clones":6,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":24,"num_read_kb":8760,"num_write":48,"num_write_kb":17808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,0,10,6,12,5],"acting":[11,8,0,10,6,12,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"99'185","reported_seq":529,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-18T23:12:15.628947+0000","last_change":"2024-04-18T23:12:09.707071+0000","last_active":"2024-04-18T23:12:15.628947+0000","last_peered":"2024-04-18T23:12:15.628947+0000","last_clean":"2024-04-18T23:12:15.628947+0000","last_became_active":"2024-04-18T23:10:54.808152+0000","last_became_peered":"2024-04-18T23:10:54.808152+0000","last_unstale":"2024-04-18T23:12:15.628947+0000","last_undegraded":"2024-04-18T23:12:15.628947+0000","last_fullsized":"2024-04-18T23:12:15.628947+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"94'175","last_scrub_stamp":"2024-04-18T23:12:03.002648+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:03.002648+0000","objects_scrubbed":14,"log_size":185,"log_dups_size":0,"ondisk_log_size":185,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.407737+0000","scrub_duration":819,"objects_trimmed":0,"snaptrim_duration":0.101814369,"stat_sum":{"num_bytes":26034176,"num_objects":16,"num_object_clones":11,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":1,"num_read":45,"num_read_kb":44229,"num_write":83,"num_write_kb":25696,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,1,12,6,7,9],"acting":[4,0,1,12,6,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"}]},{"pgid":"3.a","version":"103'36","reported_seq":392,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:19.911019+0000","last_change":"2024-04-18T23:12:09.701560+0000","last_active":"2024-04-18T23:12:19.911019+0000","last_peered":"2024-04-18T23:12:19.911019+0000","last_clean":"2024-04-18T23:12:19.911019+0000","last_became_active":"2024-04-18T23:10:54.764999+0000","last_became_peered":"2024-04-18T23:10:54.764999+0000","last_unstale":"2024-04-18T23:12:19.911019+0000","last_undegraded":"2024-04-18T23:12:19.911019+0000","last_fullsized":"2024-04-18T23:12:19.911019+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:21.710817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.102292296,"stat_sum":{"num_bytes":6619136,"num_objects":6,"num_object_clones":4,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":1,"num_read":30,"num_read_kb":4233,"num_write":13,"num_write_kb":6384,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,9,14,6,11],"acting":[3,7,1,9,14,6,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.9","version":"103'149","reported_seq":543,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:19.667040+0000","last_change":"2024-04-18T23:12:09.904267+0000","last_active":"2024-04-18T23:12:19.667040+0000","last_peered":"2024-04-18T23:12:19.667040+0000","last_clean":"2024-04-18T23:12:19.667040+0000","last_became_active":"2024-04-18T23:11:39.726824+0000","last_became_peered":"2024-04-18T23:11:39.726824+0000","last_unstale":"2024-04-18T23:12:19.667040+0000","last_undegraded":"2024-04-18T23:12:19.667040+0000","last_fullsized":"2024-04-18T23:12:19.667040+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'124","last_scrub_stamp":"2024-04-18T23:11:59.890703+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:11:59.890703+0000","objects_scrubbed":13,"log_size":149,"log_dups_size":0,"ondisk_log_size":149,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:15.511060+0000","scrub_duration":619,"objects_trimmed":0,"snaptrim_duration":0.29999762400000002,"stat_sum":{"num_bytes":5767168,"num_objects":18,"num_object_clones":12,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":1,"num_read":60,"num_read_kb":10808,"num_write":128,"num_write_kb":8128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":2867200,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,1,0,3,15,8,10],"acting":[14,1,0,3,15,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"3.8","version":"87'61","reported_seq":442,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.075144+0000","last_change":"2024-04-18T23:12:09.825833+0000","last_active":"2024-04-18T23:12:20.075144+0000","last_peered":"2024-04-18T23:12:20.075144+0000","last_clean":"2024-04-18T23:12:20.075144+0000","last_became_active":"2024-04-18T23:11:37.876250+0000","last_became_peered":"2024-04-18T23:11:37.876250+0000","last_unstale":"2024-04-18T23:12:20.075144+0000","last_undegraded":"2024-04-18T23:12:20.075144+0000","last_fullsized":"2024-04-18T23:12:20.075144+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'61","last_scrub_stamp":"2024-04-18T23:12:02.908053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:02.908053+0000","objects_scrubbed":5,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:06.062008+0000","scrub_duration":515,"objects_trimmed":0,"snaptrim_duration":0.10279075,"stat_sum":{"num_bytes":1949696,"num_objects":5,"num_object_clones":3,"num_object_copies":35,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":29,"num_read_kb":8140,"num_write":54,"num_write_kb":3312,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":262144,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,2,1,14,15,6],"acting":[5,0,2,1,14,15,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"33'157","reported_seq":388,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.670252+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:12:20.670252+0000","last_peered":"2024-04-18T23:12:20.670252+0000","last_clean":"2024-04-18T23:12:20.670252+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:12:20.670252+0000","last_undegraded":"2024-04-18T23:12:20.670252+0000","last_fullsized":"2024-04-18T23:12:20.670252+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":189,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.075239+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:12:20.075239+0000","last_peered":"2024-04-18T23:12:20.075239+0000","last_clean":"2024-04-18T23:12:20.075239+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:12:20.075239+0000","last_undegraded":"2024-04-18T23:12:20.075239+0000","last_fullsized":"2024-04-18T23:12:20.075239+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"103'109","reported_seq":482,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.670388+0000","last_change":"2024-04-18T23:12:13.461804+0000","last_active":"2024-04-18T23:12:20.670388+0000","last_peered":"2024-04-18T23:12:20.670388+0000","last_clean":"2024-04-18T23:12:20.670388+0000","last_became_active":"2024-04-18T23:11:33.818951+0000","last_became_peered":"2024-04-18T23:11:33.818951+0000","last_unstale":"2024-04-18T23:12:20.670388+0000","last_undegraded":"2024-04-18T23:12:20.670388+0000","last_fullsized":"2024-04-18T23:12:20.670388+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'109","last_scrub_stamp":"2024-04-18T23:12:13.461493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.461493+0000","objects_scrubbed":13,"log_size":109,"log_dups_size":0,"ondisk_log_size":109,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:42.351037+0000","scrub_duration":623,"objects_trimmed":0,"snaptrim_duration":0.63631145300000003,"stat_sum":{"num_bytes":17186816,"num_objects":13,"num_object_clones":9,"num_object_copies":91,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":1,"num_read":31,"num_read_kb":6014,"num_write":51,"num_write_kb":17088,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2,3,6,15,1,11],"acting":[13,2,3,6,15,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":183,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:19.910883+0000","last_change":"2024-04-18T23:12:01.049893+0000","last_active":"2024-04-18T23:12:19.910883+0000","last_peered":"2024-04-18T23:12:19.910883+0000","last_clean":"2024-04-18T23:12:19.910883+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:12:19.910883+0000","last_undegraded":"2024-04-18T23:12:19.910883+0000","last_fullsized":"2024-04-18T23:12:19.910883+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:01.049599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.049599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:09.776200+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"88'10","reported_seq":342,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-18T23:12:15.628881+0000","last_change":"2024-04-18T23:12:09.706048+0000","last_active":"2024-04-18T23:12:15.628881+0000","last_peered":"2024-04-18T23:12:15.628881+0000","last_clean":"2024-04-18T23:12:15.628881+0000","last_became_active":"2024-04-18T23:10:54.799189+0000","last_became_peered":"2024-04-18T23:10:54.799189+0000","last_unstale":"2024-04-18T23:12:15.628881+0000","last_undegraded":"2024-04-18T23:12:15.628881+0000","last_fullsized":"2024-04-18T23:12:15.628881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.101088026,"stat_sum":{"num_bytes":32768,"num_objects":2,"num_object_clones":1,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":1,"num_write":4,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,9,5,10,2,13],"acting":[4,11,9,5,10,2,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"}]},{"pgid":"2.1","version":"0'0","reported_seq":178,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.670352+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:12:20.670352+0000","last_peered":"2024-04-18T23:12:20.670352+0000","last_clean":"2024-04-18T23:12:20.670352+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:12:20.670352+0000","last_undegraded":"2024-04-18T23:12:20.670352+0000","last_fullsized":"2024-04-18T23:12:20.670352+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:21.915444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.0","version":"87'44","reported_seq":428,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:19.666990+0000","last_change":"2024-04-18T23:12:12.268091+0000","last_active":"2024-04-18T23:12:19.666990+0000","last_peered":"2024-04-18T23:12:19.666990+0000","last_clean":"2024-04-18T23:12:19.666990+0000","last_became_active":"2024-04-18T23:11:38.818022+0000","last_became_peered":"2024-04-18T23:11:38.818022+0000","last_unstale":"2024-04-18T23:12:19.666990+0000","last_undegraded":"2024-04-18T23:12:19.666990+0000","last_fullsized":"2024-04-18T23:12:19.666990+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'44","last_scrub_stamp":"2024-04-18T23:12:12.267825+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:12.267825+0000","objects_scrubbed":3,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:40.740435+0000","scrub_duration":811,"objects_trimmed":0,"snaptrim_duration":0.0011956580000000001,"stat_sum":{"num_bytes":6438912,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":20,"num_read_kb":20265,"num_write":20,"num_write_kb":6288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":7716864,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[14,2,15,1,5,6,4],"acting":[14,2,15,1,5,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":14,"acting_primary":14,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":182,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.011199+0000","last_change":"2024-04-18T23:12:14.068230+0000","last_active":"2024-04-18T23:12:20.011199+0000","last_peered":"2024-04-18T23:12:20.011199+0000","last_clean":"2024-04-18T23:12:20.011199+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:12:20.011199+0000","last_undegraded":"2024-04-18T23:12:20.011199+0000","last_fullsized":"2024-04-18T23:12:20.011199+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:14.068006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:14.068006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.692265+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":362,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.670397+0000","last_change":"2024-04-18T23:12:10.513805+0000","last_active":"2024-04-18T23:12:20.670397+0000","last_peered":"2024-04-18T23:12:20.670397+0000","last_clean":"2024-04-18T23:12:20.670397+0000","last_became_active":"2024-04-18T23:11:33.713370+0000","last_became_peered":"2024-04-18T23:11:33.713370+0000","last_unstale":"2024-04-18T23:12:20.670397+0000","last_undegraded":"2024-04-18T23:12:20.670397+0000","last_fullsized":"2024-04-18T23:12:20.670397+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:10.513548+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:10.513548+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:17.582703+0000","scrub_duration":635,"objects_trimmed":0,"snaptrim_duration":0.001054772,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,7,4,15,2,5,3],"acting":[13,7,4,15,2,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"3.5","version":"103'131","reported_seq":494,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-18T23:12:15.965050+0000","last_change":"2024-04-18T23:12:11.804773+0000","last_active":"2024-04-18T23:12:15.965050+0000","last_peered":"2024-04-18T23:12:15.965050+0000","last_clean":"2024-04-18T23:12:15.965050+0000","last_became_active":"2024-04-18T23:11:39.923911+0000","last_became_peered":"2024-04-18T23:11:39.923911+0000","last_unstale":"2024-04-18T23:12:15.965050+0000","last_undegraded":"2024-04-18T23:12:15.965050+0000","last_fullsized":"2024-04-18T23:12:15.965050+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'131","last_scrub_stamp":"2024-04-18T23:12:11.804308+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:11.804308+0000","objects_scrubbed":10,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":2,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:12.550212+0000","scrub_duration":1023,"objects_trimmed":0,"snaptrim_duration":0.209127907,"stat_sum":{"num_bytes":8749056,"num_objects":10,"num_object_clones":7,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":41,"num_read_kb":12303,"num_write":64,"num_write_kb":14000,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":5685248,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,8,4,13,7,0,6],"acting":[15,8,4,13,7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"}]},{"pgid":"2.4","version":"0'0","reported_seq":176,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-18T23:12:15.629119+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:12:15.629119+0000","last_peered":"2024-04-18T23:12:15.629119+0000","last_clean":"2024-04-18T23:12:15.629119+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:12:15.629119+0000","last_undegraded":"2024-04-18T23:12:15.629119+0000","last_fullsized":"2024-04-18T23:12:15.629119+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.4","version":"103'123","reported_seq":444,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-18T23:12:15.975859+0000","last_change":"2024-04-18T23:12:10.010326+0000","last_active":"2024-04-18T23:12:15.975859+0000","last_peered":"2024-04-18T23:12:15.975859+0000","last_clean":"2024-04-18T23:12:15.975859+0000","last_became_active":"2024-04-18T23:10:54.793319+0000","last_became_peered":"2024-04-18T23:10:54.793319+0000","last_unstale":"2024-04-18T23:12:15.975859+0000","last_undegraded":"2024-04-18T23:12:15.975859+0000","last_fullsized":"2024-04-18T23:12:15.975859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:10:53.417614+0000","objects_scrubbed":0,"log_size":123,"log_dups_size":0,"ondisk_log_size":123,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.40830994700000001,"stat_sum":{"num_bytes":15892480,"num_objects":11,"num_object_clones":7,"num_object_copies":77,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":1,"num_read":18,"num_read_kb":9,"num_write":42,"num_write_kb":14224,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,4,1,0,14,13],"acting":[8,2,4,1,0,14,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"}]},{"pgid":"2.5","version":"0'0","reported_seq":184,"reported_epoch":109,"state":"active+clean","last_fresh":"2024-04-18T23:12:19.975970+0000","last_change":"2024-04-18T23:12:13.697974+0000","last_active":"2024-04-18T23:12:19.975970+0000","last_peered":"2024-04-18T23:12:19.975970+0000","last_clean":"2024-04-18T23:12:19.975970+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:12:19.975970+0000","last_undegraded":"2024-04-18T23:12:19.975970+0000","last_fullsized":"2024-04-18T23:12:19.975970+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.697732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.697732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:38.678587+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.7","version":"103'254","reported_seq":582,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-18T23:12:15.629138+0000","last_change":"2024-04-18T23:12:13.864427+0000","last_active":"2024-04-18T23:12:15.629138+0000","last_peered":"2024-04-18T23:12:15.629138+0000","last_clean":"2024-04-18T23:12:15.629138+0000","last_became_active":"2024-04-18T23:10:54.693609+0000","last_became_peered":"2024-04-18T23:10:54.693609+0000","last_unstale":"2024-04-18T23:12:15.629138+0000","last_undegraded":"2024-04-18T23:12:15.629138+0000","last_fullsized":"2024-04-18T23:12:15.629138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"103'254","last_scrub_stamp":"2024-04-18T23:12:13.863947+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.863947+0000","objects_scrubbed":22,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:43.811365+0000","scrub_duration":823,"objects_trimmed":0,"snaptrim_duration":0.30498107400000002,"stat_sum":{"num_bytes":25444352,"num_objects":22,"num_object_clones":14,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":1,"num_read":52,"num_read_kb":11706,"num_write":186,"num_write_kb":27360,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,0,8,5,11],"acting":[4,6,9,0,8,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"23","length":"1"},{"start":"25","length":"1"}]},{"pgid":"2.6","version":"0'0","reported_seq":175,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-18T23:12:16.076024+0000","last_change":"2024-04-18T23:12:15.125832+0000","last_active":"2024-04-18T23:12:16.076024+0000","last_peered":"2024-04-18T23:12:16.076024+0000","last_clean":"2024-04-18T23:12:16.076024+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:12:16.076024+0000","last_undegraded":"2024-04-18T23:12:16.076024+0000","last_fullsized":"2024-04-18T23:12:16.076024+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_clean_scrub_stamp":"2024-04-18T23:12:15.125684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.727019+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.6","version":"87'42","reported_seq":432,"reported_epoch":110,"state":"active+clean","last_fresh":"2024-04-18T23:12:20.669476+0000","last_change":"2024-04-18T23:12:12.042113+0000","last_active":"2024-04-18T23:12:20.669476+0000","last_peered":"2024-04-18T23:12:20.669476+0000","last_clean":"2024-04-18T23:12:20.669476+0000","last_became_active":"2024-04-18T23:11:39.717443+0000","last_became_peered":"2024-04-18T23:11:39.717443+0000","last_unstale":"2024-04-18T23:12:20.669476+0000","last_undegraded":"2024-04-18T23:12:20.669476+0000","last_fullsized":"2024-04-18T23:12:20.669476+0000","mapping_epoch":72,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":73,"parent":"0.0","parent_split_bits":0,"last_scrub":"87'42","last_scrub_stamp":"2024-04-18T23:12:12.041816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:53.417614+0000","last_clean_scrub_stamp":"2024-04-18T23:12:12.041816+0000","objects_scrubbed":3,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:33.744302+0000","scrub_duration":847,"objects_trimmed":0,"snaptrim_duration":0.00078749699999999996,"stat_sum":{"num_bytes":3358720,"num_objects":3,"num_object_clones":1,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":13209,"num_write":18,"num_write_kb":4928,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":1802240,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,13,1,15,12,7,9],"acting":[0,13,1,15,12,7,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":102,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-18T23:12:15.965018+0000","last_change":"2024-04-18T23:12:13.759380+0000","last_active":"2024-04-18T23:12:15.965018+0000","last_peered":"2024-04-18T23:12:15.965018+0000","last_clean":"2024-04-18T23:12:15.965018+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:12:15.965018+0000","last_undegraded":"2024-04-18T23:12:15.965018+0000","last_fullsized":"2024-04-18T23:12:15.965018+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.758998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.758998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:41.357877+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":16,"stat_sum":{"num_bytes":172572672,"num_objects":142,"num_object_clones":92,"num_object_copies":994,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":142,"num_whiteouts":7,"num_read":480,"num_read_kb":187374,"num_write":898,"num_write_kb":186720,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":22,"num_bytes_recovered":18333696,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":298364928,"data_stored":457891840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0},"log_size":1512,"ondisk_log_size":1512,"up":112,"acting":112,"num_store_stats":16},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808842,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40776,"kb_used_data":12220,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94331064,"statfs":{"total":96636764160,"available":96595009536,"internally_reserved":0,"allocated":12513280,"data_stored":19248154,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280536,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46808,"kb_used_data":15628,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94325032,"statfs":{"total":96636764160,"available":96588832768,"internally_reserved":0,"allocated":16003072,"data_stored":21344002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280536,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44952,"kb_used_data":13768,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94326888,"statfs":{"total":96636764160,"available":96590733312,"internally_reserved":0,"allocated":14098432,"data_stored":19875760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313239,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63472,"kb_used_data":32288,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94308368,"statfs":{"total":96636764160,"available":96571768832,"internally_reserved":0,"allocated":33062912,"data_stored":48022928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313239,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56136,"kb_used_data":24948,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94315704,"statfs":{"total":96636764160,"available":96579280896,"internally_reserved":0,"allocated":25546752,"data_stored":40161026,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313240,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40320,"kb_used_data":13292,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94331520,"statfs":{"total":96636764160,"available":96595476480,"internally_reserved":0,"allocated":13611008,"data_stored":18575106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313239,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56872,"kb_used_data":25692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94314968,"statfs":{"total":96636764160,"available":96578527232,"internally_reserved":0,"allocated":26308608,"data_stored":38937626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313240,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":41164,"kb_used_data":14144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94330676,"statfs":{"total":96636764160,"available":96594612224,"internally_reserved":0,"allocated":14483456,"data_stored":19447554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313237,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59272,"kb_used_data":28096,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94312568,"statfs":{"total":96636764160,"available":96576069632,"internally_reserved":0,"allocated":28770304,"data_stored":42020192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313238,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53728,"kb_used_data":22548,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94318112,"statfs":{"total":96636764160,"available":96581746688,"internally_reserved":0,"allocated":23089152,"data_stored":33206018,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313239,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47808,"kb_used_data":16624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324032,"statfs":{"total":96636764160,"available":96587808768,"internally_reserved":0,"allocated":17022976,"data_stored":25404816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280536,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47580,"kb_used_data":16400,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94324260,"statfs":{"total":96636764160,"available":96588042240,"internally_reserved":0,"allocated":16793600,"data_stored":24178978,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280536,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":81204,"kb_used_data":31976,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94290636,"statfs":{"total":96636764160,"available":96553611264,"internally_reserved":0,"allocated":32743424,"data_stored":50781954,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280536,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":48136,"kb_used_data":16952,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94323704,"statfs":{"total":96636764160,"available":96587472896,"internally_reserved":0,"allocated":17358848,"data_stored":25115348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280535,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51252,"kb_used_data":20072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94320588,"statfs":{"total":96636764160,"available":96584282112,"internally_reserved":0,"allocated":20553728,"data_stored":31327258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280535,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44016,"kb_used_data":12836,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94327824,"statfs":{"total":96636764160,"available":96591691776,"internally_reserved":0,"allocated":13144064,"data_stored":16426010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":31420416,"data_stored":47054848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":23920640,"data_stored":39202816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":14635008,"data_stored":22691840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11984896,"data_stored":17616896,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":24748032,"data_stored":38019072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12857344,"data_stored":18489344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":31117312,"data_stored":49823744,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":15781888,"data_stored":24186880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":27226112,"data_stored":41111552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":18993152,"data_stored":30408704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":11583488,"data_stored":15507456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":21463040,"data_stored":32247808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":15380480,"data_stored":24436736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":11923456,"data_stored":18378752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":14,"total":0,"available":0,"internally_reserved":0,"allocated":14376960,"data_stored":20385792,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":10952704,"data_stored":18329600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":3761,"internal_metadata":0}]}} 2024-04-18T23:12:21.974 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2024-04-18T23:12:21.974 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2024-04-18T23:12:22.032 INFO:tasks.ceph.osd.6.smithi136.stderr:2024-04-18T23:12:22.032+0000 7f54bc565640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 25062) UID: 0 2024-04-18T23:12:22.103 INFO:tasks.ceph.osd.13.smithi120.stderr:2024-04-18T23:12:22.097+0000 7f2655963640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 13 (PID: 25060) UID: 0 2024-04-18T23:12:22.113 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:12:22.113 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:12:22.113 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:22.127 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.10.asok dump_historic_ops 2024-04-18T23:12:22.203 INFO:tasks.ceph.osd.4.smithi073.stderr:2024-04-18T23:12:22.198+0000 7f9eb27d8640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 25175) UID: 0 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout: "size": 20, 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 600, 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [ 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 61/31 ECSubWrite(tid=70, reqid=osd.7.0:70, at_version=61'48, trim_to=0'0, roll_forward_to=58'46))", 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.534285830999998, 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30382444400000003, 2024-04-18T23:12:22.287 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.288 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751941+0000", 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.68e-07 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751960+0000", 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8916e-05 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751973+0000", 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3030999999999999e-05 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.289 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.751984+0000", 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0858e-05 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.853770+0000", 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.101786394 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.853799+0000", 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8918999999999999e-05 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.290 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055725+0000", 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20192524000000001 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055766+0000", 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.0917999999999997e-05 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 61/31 ECSubWrite(tid=71, reqid=osd.7.0:71, at_version=61'50, trim_to=0'0, roll_forward_to=58'46))", 2024-04-18T23:12:22.291 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 55.532742335000002, 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30231752299999998, 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.292 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753485+0000", 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2699999999999999e-07 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753510+0000", 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4779000000000001e-05 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753524+0000", 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4423999999999999e-05 2024-04-18T23:12:22.293 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.753534+0000", 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.7089999999999997e-06 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.954802+0000", 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20126760199999999 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:26.954819+0000", 2024-04-18T23:12:22.294 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7501999999999999e-05 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055781+0000", 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10096224299999999 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:27.055802+0000", 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.0636999999999999e-05 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.295 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 72/31 ECSubWrite(tid=63, reqid=osd.11.0:63, at_version=72'32, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.837897+0000", 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.448329684000001, 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203598052, 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837897+0000", 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.296 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837897+0000", 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837898+0000", 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8700000000000005e-07 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837972+0000", 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.4115000000000004e-05 2024-04-18T23:12:22.297 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.837995+0000", 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2589999999999999e-05 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838014+0000", 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8879e-05 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.938244+0000", 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100230344 2024-04-18T23:12:22.298 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.938265+0000", 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1396e-05 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041423+0000", 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10315798700000001 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041495+0000", 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.1953999999999996e-05 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.299 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 72/31 ECSubWrite(tid=64, reqid=client.4568.0:562, at_version=72'33, trim_to=0'0, roll_forward_to=71'27))", 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:38.838056+0000", 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 43.448170937, 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203492644, 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.300 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838056+0000", 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838056+0000", 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838057+0000", 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.3399999999999997e-07 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.301 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838308+0000", 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025151500000000001 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838327+0000", 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8638999999999999e-05 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:38.838339+0000", 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.1950000000000001e-05 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.302 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.039312+0000", 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20097348100000001 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.039330+0000", 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7672e-05 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041507+0000", 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0021768360000000001 2024-04-18T23:12:22.303 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:39.041549+0000", 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.1616999999999998e-05 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 82/72 ECSubWrite(tid=194, reqid=client.4568.0:674, at_version=82'98, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.661023+0000", 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.625204046999997, 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30022358900000001, 2024-04-18T23:12:22.304 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661023+0000", 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661023+0000", 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.305 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661024+0000", 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.6599999999999994e-07 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661197+0000", 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00017270400000000001 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661218+0000", 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1543e-05 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.306 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.661229+0000", 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0539e-05 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.858998+0000", 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.197769312 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.859008+0000", 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0365e-05 2024-04-18T23:12:22.307 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961168+0000", 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10215980500000001 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961246+0000", 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.8355000000000005e-05 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 82/72 ECSubWrite(tid=195, reqid=client.4568.0:675, at_version=82'99, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:12:22.308 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.760788+0000", 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.525439218000002, 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20119051900000001, 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.760788+0000", 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.760788+0000", 2024-04-18T23:12:22.309 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.760790+0000", 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1160000000000002e-06 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761152+0000", 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00036268500000000001 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761191+0000", 2024-04-18T23:12:22.310 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8374000000000002e-05 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.761221+0000", 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0431999999999999e-05 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.859990+0000", 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.098768442999999997 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.860005+0000", 2024-04-18T23:12:22.311 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5036e-05 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961955+0000", 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10195014400000001 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961978+0000", 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3289e-05 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.312 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 82/72 ECSubWrite(tid=196, reqid=client.4568.0:676, at_version=82'100, trim_to=0'0, roll_forward_to=81'96))", 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:48.762849+0000", 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 33.523378088999998, 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.199319412, 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762849+0000", 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.313 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762849+0000", 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.762851+0000", 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.81e-06 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.763087+0000", 2024-04-18T23:12:22.314 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000236512 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.763123+0000", 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.6171000000000003e-05 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.763151+0000", 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7212e-05 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.315 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961317+0000", 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19816698099999999 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.961328+0000", 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0524e-05 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.962142+0000", 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00081420799999999997 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.316 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:48.962168+0000", 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5993999999999999e-05 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 84/31 ECSubWrite(tid=148, reqid=client.4568.0:750, at_version=84'124, trim_to=0'0, roll_forward_to=83'121))", 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:11:50.980774+0000", 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 31.305453072999999, 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.201355073, 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.317 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.980774+0000", 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.980774+0000", 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.318 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.980775+0000", 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5370000000000001e-06 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.981058+0000", 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00028255700000000001 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.981081+0000", 2024-04-18T23:12:22.319 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.3535000000000001e-05 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:50.981103+0000", 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1265999999999999e-05 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.179112+0000", 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.198009659 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.179140+0000", 2024-04-18T23:12:22.320 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.8008999999999999e-05 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.182090+0000", 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029498990000000002 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:11:51.182129+0000", 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.8611e-05 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.321 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=174, reqid=client.4568.0:907, at_version=98'151, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.683135+0000", 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.603091906, 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30320944399999999, 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683135+0000", 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.322 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683135+0000", 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683136+0000", 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683285+0000", 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000149352 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.323 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.683304+0000", 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8508999999999998e-05 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783382+0000", 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10007812000000001 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783518+0000", 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00013595999999999999 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.324 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783549+0000", 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.1309000000000001e-05 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986291+0000", 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202741639 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986344+0000", 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.3594999999999998e-05 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.325 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=175, reqid=client.4568.0:908, at_version=98'152, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.783421+0000", 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.502806004, 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20296605300000001, 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783421+0000", 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.326 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783421+0000", 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783422+0000", 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.2699999999999998e-07 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.327 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783585+0000", 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000163673 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783611+0000", 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5644999999999999e-05 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783626+0000", 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4430000000000001e-05 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.784619+0000", 2024-04-18T23:12:22.328 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099339299999999997 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.784645+0000", 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5896000000000001e-05 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986354+0000", 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20170896799999999 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986387+0000", 2024-04-18T23:12:22.329 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3120999999999997e-05 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=176, reqid=client.4568.0:909, at_version=98'153, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.783642+0000", 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.502584767999998, 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20277439999999999, 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.330 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783642+0000", 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783642+0000", 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783643+0000", 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.89e-07 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.331 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783751+0000", 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000108342 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783760+0000", 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.414e-06 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783767+0000", 2024-04-18T23:12:22.332 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.216e-06 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.785732+0000", 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0019647110000000001 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.785759+0000", 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7064999999999999e-05 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.333 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986391+0000", 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20063127 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986417+0000", 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.5993e-05 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 98/31 ECSubWrite(tid=177, reqid=client.4568.0:910, at_version=98'154, trim_to=0'0, roll_forward_to=87'148))", 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.783778+0000", 2024-04-18T23:12:22.334 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.502448668, 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203022498, 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783778+0000", 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783778+0000", 2024-04-18T23:12:22.335 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783779+0000", 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.27e-07 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783797+0000", 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8011000000000001e-05 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.336 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783804+0000", 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.7049999999999999e-06 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.783811+0000", 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.2519999999999996e-06 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.985869+0000", 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20205835 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.337 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.985886+0000", 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.7359000000000001e-05 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986768+0000", 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00088193500000000003 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.986801+0000", 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.2558999999999998e-05 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.338 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpRead(3.ds3 98/31 ECSubRead(tid=183, to_read={3:b7d0d2d3:::smithi14230322-10:head=0,647168,0}, subchunks={3:b7d0d2d3:::smithi14230322-10:head=[0,1]}, attrs_to_read=))", 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:04.988115+0000", 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 17.298112009, 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20139596500000001, 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988115+0000", 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.339 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988115+0000", 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988115+0000", 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.3799999999999998e-07 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988124+0000", 2024-04-18T23:12:22.340 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.242e-06 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:04.988137+0000", 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3468e-05 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:05.088254+0000", 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100116893 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.341 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:05.188513+0000", 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100258602 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:05.189511+0000", 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00099832200000000005 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.9s6 99/72 ECSubWrite(tid=255, reqid=osd.14.0:254, at_version=0'0, trim_to=0'0, roll_forward_to=99'145))", 2024-04-18T23:12:22.342 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.499438+0000", 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.786789325999999, 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.19688966199999999, 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.499438+0000", 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.343 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.499438+0000", 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.499441+0000", 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3060000000000002e-06 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.499477+0000", 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.5778999999999997e-05 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.344 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.499522+0000", 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.5652000000000003e-05 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.599618+0000", 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10009605000000001 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.599874+0000", 2024-04-18T23:12:22.345 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00025543000000000001 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.599887+0000", 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.2967e-05 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.696258+0000", 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.096371471 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.346 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.696327+0000", 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.9006999999999998e-05 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 100/31 ECSubWrite(tid=190, reqid=client.4568.0:945, at_version=100'163, trim_to=0'0, roll_forward_to=99'161))", 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.593564+0000", 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.692663344, 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.30472582100000001, 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.347 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.593564+0000", 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.593564+0000", 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.348 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.593564+0000", 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 4.2599999999999998e-07 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.594326+0000", 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.00076163800000000003 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.594353+0000", 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.7662e-05 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.349 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694522+0000", 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100168681 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.794787+0000", 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100265391 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.794818+0000", 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0392999999999999e-05 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.350 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898202+0000", 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103384437 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898289+0000", 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 8.7193000000000004e-05 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 100/31 ECSubWrite(tid=191, reqid=client.4568.0:946, at_version=100'164, trim_to=0'0, roll_forward_to=99'161))", 2024-04-18T23:12:22.351 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.694580+0000", 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.591647359, 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20384158099999999, 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694580+0000", 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694580+0000", 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.352 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694581+0000", 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.5999999999999991e-07 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694830+0000", 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000249917 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694855+0000", 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.4511999999999998e-05 2024-04-18T23:12:22.353 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694869+0000", 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.36e-05 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.797475+0000", 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.102606054 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.797496+0000", 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.1064999999999999e-05 2024-04-18T23:12:22.354 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898355+0000", 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.10085888899999999 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898421+0000", 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.6583999999999998e-05 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.355 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 100/31 ECSubWrite(tid=192, reqid=client.4568.0:947, at_version=100'165, trim_to=0'0, roll_forward_to=99'161))", 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.694893+0000", 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.591334284, 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20358923300000001, 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694893+0000", 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.356 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694893+0000", 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.694893+0000", 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 7.2600000000000002e-07 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695132+0000", 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.000238211 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.357 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695148+0000", 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.6203000000000001e-05 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695158+0000", 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.9490000000000008e-06 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.798427+0000", 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.103269048 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.358 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.798443+0000", 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.5869e-05 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898428+0000", 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099984956 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898482+0000", 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.4271000000000003e-05 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.359 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 100/31 ECSubWrite(tid=193, reqid=client.4568.0:948, at_version=100'166, trim_to=0'0, roll_forward_to=99'161))", 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.695174+0000", 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.591053328999999, 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.203367723, 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695174+0000", 2024-04-18T23:12:22.360 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695174+0000", 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695174+0000", 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.2699999999999999e-07 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.361 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695232+0000", 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.7948000000000002e-05 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695247+0000", 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4812e-05 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695258+0000", 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.0997e-05 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.362 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.799394+0000", 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.104136144 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.799409+0000", 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4683e-05 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898486+0000", 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.099077083999999996 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.363 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898541+0000", 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.5528000000000001e-05 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.ds3 100/31 ECSubWrite(tid=194, reqid=client.4568.0:949, at_version=100'167, trim_to=0'0, roll_forward_to=99'161))", 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.695271+0000", 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.590955499, 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.20368275399999999, 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.364 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695271+0000", 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695271+0000", 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695272+0000", 2024-04-18T23:12:22.365 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 5.06e-07 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695287+0000", 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.4797000000000001e-05 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695300+0000", 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.3076e-05 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.695309+0000", 2024-04-18T23:12:22.366 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.1200000000000008e-06 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.897554+0000", 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.202244749 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.897584+0000", 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.0392e-05 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.367 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898888+0000", 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0013043659999999999 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.898954+0000", 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 6.5747999999999995e-05 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: "description": "MOSDECSubOpWrite(3.cs3 100/31 ECSubWrite(tid=107, reqid=client.4568.0:943, at_version=100'69, trim_to=0'0, roll_forward_to=99'67))", 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: "initiated_at": "2024-04-18T23:12:06.920934+0000", 2024-04-18T23:12:22.368 INFO:teuthology.orchestra.run.smithi136.stdout: "age": 15.365292896, 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.11068270700000001, 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "continuous": false, 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "type_data": { 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "flag_point": "reached pg", 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "events": [ 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "initiated", 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.920934+0000", 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "header_read", 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.920934+0000", 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0 2024-04-18T23:12:22.369 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "throttled", 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.920934+0000", 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.9299999999999999e-07 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "all_read", 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.923907+0000", 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0029729650000000002 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "dispatched", 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.923941+0000", 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 3.3439000000000003e-05 2024-04-18T23:12:22.370 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "queued_for_pg", 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:06.923960+0000", 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 1.8709e-05 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "reached_pg", 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:07.024269+0000", 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.100309971 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_started", 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:07.024292+0000", 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 2.2552999999999999e-05 2024-04-18T23:12:22.371 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "sub_op_committed", 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:07.031526+0000", 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 0.0072339910000000004 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: }, 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: { 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: "event": "done", 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: "time": "2024-04-18T23:12:07.031617+0000", 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: "duration": 9.0686000000000006e-05 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: } 2024-04-18T23:12:22.372 INFO:teuthology.orchestra.run.smithi136.stdout: ] 2024-04-18T23:12:22.373 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:22.374 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_ops_in_flight 2024-04-18T23:12:22.375 INFO:tasks.ceph.osd.15.smithi142.stderr:2024-04-18T23:12:22.374+0000 7f742801a640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 15 (PID: 33520) UID: 0 2024-04-18T23:12:22.414 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-18T23:12:22.474 INFO:tasks.ceph.osd.2.smithi136.stderr:2024-04-18T23:12:22.472+0000 7f32a4b33640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 25060) UID: 0 2024-04-18T23:12:22.500 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:22.501 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:22.501 INFO:teuthology.orchestra.run.smithi136.stdout: "num_ops": 0 2024-04-18T23:12:22.501 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:22.515 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_blocked_ops 2024-04-18T23:12:22.559 INFO:teuthology.orchestra.run.smithi073.stdout:osd.0: {} 2024-04-18T23:12:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-18T23:12:22.579 INFO:teuthology.orchestra.run.smithi073.stdout:osd.1: {} 2024-04-18T23:12:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-18T23:12:22.597 INFO:teuthology.orchestra.run.smithi073.stdout:osd.2: {} 2024-04-18T23:12:22.597 INFO:teuthology.orchestra.run.smithi073.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-18T23:12:22.600 INFO:tasks.ceph.osd.10.smithi136.stderr:2024-04-18T23:12:22.600+0000 7f6608eae640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 10 (PID: 25064) UID: 0 2024-04-18T23:12:22.617 INFO:teuthology.orchestra.run.smithi073.stdout:osd.3: {} 2024-04-18T23:12:22.617 INFO:teuthology.orchestra.run.smithi073.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-18T23:12:22.633 INFO:teuthology.orchestra.run.smithi073.stdout:osd.4: {} 2024-04-18T23:12:22.633 INFO:teuthology.orchestra.run.smithi073.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-18T23:12:22.652 INFO:teuthology.orchestra.run.smithi073.stdout:osd.5: {} 2024-04-18T23:12:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-18T23:12:22.671 INFO:teuthology.orchestra.run.smithi073.stdout:osd.6: {} 2024-04-18T23:12:22.671 INFO:teuthology.orchestra.run.smithi073.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-18T23:12:22.677 INFO:tasks.ceph.osd.7.smithi142.stderr:2024-04-18T23:12:22.674+0000 7f9f7e5ce640 -1 received signal: Hangup from /usr/bin/python3 /usr/bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 25339) UID: 0 2024-04-18T23:12:22.681 INFO:teuthology.orchestra.run.smithi136.stdout:{ 2024-04-18T23:12:22.681 INFO:teuthology.orchestra.run.smithi136.stdout: "ops": [], 2024-04-18T23:12:22.681 INFO:teuthology.orchestra.run.smithi136.stdout: "complaint_time": 30, 2024-04-18T23:12:22.681 INFO:teuthology.orchestra.run.smithi136.stdout: "num_blocked_ops": 0 2024-04-18T23:12:22.681 INFO:teuthology.orchestra.run.smithi136.stdout:} 2024-04-18T23:12:22.686 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'unique_pool_0' removed 2024-04-18T23:12:22.687 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.7 to 1 (10000) 2024-04-18T23:12:22.693 INFO:teuthology.orchestra.run.smithi073.stdout:osd.7: {} 2024-04-18T23:12:22.693 INFO:teuthology.orchestra.run.smithi073.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-18T23:12:22.697 DEBUG:teuthology.orchestra.run.smithi136:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.14.asok dump_historic_ops 2024-04-18T23:12:22.709 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-18T23:12:22.719 INFO:tasks.thrashosds:joining thrashosds 2024-04-18T23:12:22.719 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-04-18T23:12:22.720 INFO:teuthology.orchestra.run.smithi073.stdout:osd.8: {} 2024-04-18T23:12:22.720 INFO:teuthology.orchestra.run.smithi073.stderr:osd.8: osd_enable_op_tracker = '' 2024-04-18T23:12:22.731 INFO:teuthology.orchestra.run.smithi073.stdout:osd.9: {} 2024-04-18T23:12:22.732 INFO:teuthology.orchestra.run.smithi073.stderr:osd.9: osd_enable_op_tracker = '' 2024-04-18T23:12:22.751 INFO:teuthology.orchestra.run.smithi073.stdout:osd.10: {} 2024-04-18T23:12:22.751 INFO:teuthology.orchestra.run.smithi073.stderr:osd.10: osd_enable_op_tracker = '' 2024-04-18T23:12:22.773 INFO:teuthology.orchestra.run.smithi073.stdout:osd.11: {} 2024-04-18T23:12:22.774 INFO:teuthology.orchestra.run.smithi073.stderr:osd.11: osd_enable_op_tracker = '' 2024-04-18T23:12:22.804 INFO:teuthology.orchestra.run.smithi073.stdout:osd.12: {} 2024-04-18T23:12:22.804 INFO:teuthology.orchestra.run.smithi073.stderr:osd.12: osd_enable_op_tracker = '' 2024-04-18T23:12:22.824 INFO:teuthology.orchestra.run.smithi073.stdout:osd.13: {} 2024-04-18T23:12:22.824 INFO:teuthology.orchestra.run.smithi073.stderr:osd.13: osd_enable_op_tracker = '' 2024-04-18T23:12:22.846 INFO:teuthology.orchestra.run.smithi073.stdout:osd.14: {} 2024-04-18T23:12:22.846 INFO:teuthology.orchestra.run.smithi073.stderr:osd.14: osd_enable_op_tracker = '' 2024-04-18T23:12:22.862 INFO:teuthology.orchestra.run.smithi136.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:22.862 INFO:teuthology.orchestra.run.smithi136.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:22.866 INFO:teuthology.orchestra.run.smithi073.stdout:osd.15: {} 2024-04-18T23:12:22.867 INFO:teuthology.orchestra.run.smithi073.stderr:osd.15: osd_enable_op_tracker = '' 2024-04-18T23:12:22.877 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-18T23:12:23.006 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:23.006 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:23.020 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-18T23:12:23.142 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:23.143 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:23.159 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-18T23:12:23.278 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:23.278 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:23.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-18T23:12:23.417 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:23.417 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:23.431 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-18T23:12:23.549 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:23.549 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:23.564 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-18T23:12:23.685 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:23.685 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:23.701 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_ops_in_flight 2024-04-18T23:12:23.701 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.8 to 1 (10000) 2024-04-18T23:12:23.727 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-04-18T23:12:23.800 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-18T23:12:23.823 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:23.823 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:23.837 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_blocked_ops 2024-04-18T23:12:23.959 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:23.960 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:23.975 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.11.asok dump_historic_ops 2024-04-18T23:12:24.103 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:24.103 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:24.116 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_ops_in_flight 2024-04-18T23:12:24.233 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:24.233 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:24.247 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_blocked_ops 2024-04-18T23:12:24.363 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:24.363 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:24.377 DEBUG:teuthology.orchestra.run.smithi142:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.15.asok dump_historic_ops 2024-04-18T23:12:24.500 INFO:teuthology.orchestra.run.smithi142.stdout:ERROR: (22) Invalid argument 2024-04-18T23:12:24.500 INFO:teuthology.orchestra.run.smithi142.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-18T23:12:24.713 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.9 to 1 (10000) 2024-04-18T23:12:24.713 INFO:teuthology.orchestra.run.smithi073.stderr:noscrub is unset 2024-04-18T23:12:24.738 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-18T23:12:24.739 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-04-18T23:12:25.732 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.10 to 1 (10000) 2024-04-18T23:12:25.732 INFO:teuthology.orchestra.run.smithi073.stderr:nodeep-scrub is unset 2024-04-18T23:12:25.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-04-18T23:12:26.731 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.11 to 1 (10000) 2024-04-18T23:12:26.756 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-04-18T23:12:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.12 to 1 (10000) 2024-04-18T23:12:27.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-04-18T23:12:28.748 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.13 to 1 (10000) 2024-04-18T23:12:28.773 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-04-18T23:12:29.761 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.14 to 1 (10000) 2024-04-18T23:12:29.786 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-04-18T23:12:30.771 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.15 to 1 (10000) 2024-04-18T23:12:30.797 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-18T23:12:30.797 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:12:31.101 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:12:31.101 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:12:31.119 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":159,"stamp":"2024-04-18T23:12:30.767663+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":542328,"kb_used_data":27968,"kb_used_omap":35,"kb_used_meta":514076,"kb_avail":1509407112,"statfs":{"total":1546188226560,"available":1545632882688,"internally_reserved":0,"allocated":28639232,"data_stored":17275346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":526414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040480"},"pg_stats":[{"pgid":"1.0","version":"33'157","reported_seq":397,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.856474+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:12:25.856474+0000","last_peered":"2024-04-18T23:12:25.856474+0000","last_clean":"2024-04-18T23:12:25.856474+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:12:25.856474+0000","last_undegraded":"2024-04-18T23:12:25.856474+0000","last_fullsized":"2024-04-18T23:12:25.856474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.849949+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:12:25.849949+0000","last_peered":"2024-04-18T23:12:25.849949+0000","last_clean":"2024-04-18T23:12:25.849949+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:12:25.849949+0000","last_undegraded":"2024-04-18T23:12:25.849949+0000","last_fullsized":"2024-04-18T23:12:25.849949+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.754968+0000","last_change":"2024-04-18T23:12:01.049893+0000","last_active":"2024-04-18T23:12:25.754968+0000","last_peered":"2024-04-18T23:12:25.754968+0000","last_clean":"2024-04-18T23:12:25.754968+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:12:25.754968+0000","last_undegraded":"2024-04-18T23:12:25.754968+0000","last_fullsized":"2024-04-18T23:12:25.754968+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:01.049599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.049599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:09.776200+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":187,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.956577+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:12:25.956577+0000","last_peered":"2024-04-18T23:12:25.956577+0000","last_clean":"2024-04-18T23:12:25.956577+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:12:25.956577+0000","last_undegraded":"2024-04-18T23:12:25.956577+0000","last_fullsized":"2024-04-18T23:12:25.956577+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":194,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.754888+0000","last_change":"2024-04-18T23:12:14.068230+0000","last_active":"2024-04-18T23:12:25.754888+0000","last_peered":"2024-04-18T23:12:25.754888+0000","last_clean":"2024-04-18T23:12:25.754888+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:12:25.754888+0000","last_undegraded":"2024-04-18T23:12:25.754888+0000","last_fullsized":"2024-04-18T23:12:25.754888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:14.068006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:14.068006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.692265+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":194,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:26.015487+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:12:26.015487+0000","last_peered":"2024-04-18T23:12:26.015487+0000","last_clean":"2024-04-18T23:12:26.015487+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:12:26.015487+0000","last_undegraded":"2024-04-18T23:12:26.015487+0000","last_fullsized":"2024-04-18T23:12:26.015487+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":195,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.955497+0000","last_change":"2024-04-18T23:12:13.697974+0000","last_active":"2024-04-18T23:12:25.955497+0000","last_peered":"2024-04-18T23:12:25.955497+0000","last_clean":"2024-04-18T23:12:25.955497+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:12:25.955497+0000","last_undegraded":"2024-04-18T23:12:25.955497+0000","last_fullsized":"2024-04-18T23:12:25.955497+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.697732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.697732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:38.678587+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":194,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.753532+0000","last_change":"2024-04-18T23:12:15.125832+0000","last_active":"2024-04-18T23:12:25.753532+0000","last_peered":"2024-04-18T23:12:25.753532+0000","last_clean":"2024-04-18T23:12:25.753532+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:12:25.753532+0000","last_undegraded":"2024-04-18T23:12:25.753532+0000","last_fullsized":"2024-04-18T23:12:25.753532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_clean_scrub_stamp":"2024-04-18T23:12:15.125684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.727019+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.756438+0000","last_change":"2024-04-18T23:12:13.759380+0000","last_active":"2024-04-18T23:12:25.756438+0000","last_peered":"2024-04-18T23:12:25.756438+0000","last_clean":"2024-04-18T23:12:25.756438+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:12:25.756438+0000","last_undegraded":"2024-04-18T23:12:25.756438+0000","last_fullsized":"2024-04-18T23:12:25.756438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.758998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.758998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:41.357877+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808844,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30244,"kb_used_data":1688,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94341596,"statfs":{"total":96636764160,"available":96605794304,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280537,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1545488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313240,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313240,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32836,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1695744,"data_stored":997572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313241,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313241,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313241,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1695744,"data_stored":997572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313239,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313239,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32836,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1695744,"data_stored":997572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313240,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280537,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1545488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280537,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50928,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320912,"statfs":{"total":96636764160,"available":96584613888,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280538,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32888,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1744896,"data_stored":1026054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:12:31.121 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:12:31.411 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:12:31.412 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:12:31.430 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-18T23:12:31.296791+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":542328,"kb_used_data":27968,"kb_used_omap":35,"kb_used_meta":514076,"kb_avail":1509407112,"statfs":{"total":1546188226560,"available":1545632882688,"internally_reserved":0,"allocated":28639232,"data_stored":17275346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":526414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.001114"},"pg_stats":[{"pgid":"1.0","version":"33'157","reported_seq":397,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.856474+0000","last_change":"2024-04-18T23:11:47.941542+0000","last_active":"2024-04-18T23:12:25.856474+0000","last_peered":"2024-04-18T23:12:25.856474+0000","last_clean":"2024-04-18T23:12:25.856474+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:12:25.856474+0000","last_undegraded":"2024-04-18T23:12:25.856474+0000","last_fullsized":"2024-04-18T23:12:25.856474+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:11:47.941375+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.941375+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:04.901450+0000","scrub_duration":175,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":199,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.849949+0000","last_change":"2024-04-18T23:11:47.796840+0000","last_active":"2024-04-18T23:12:25.849949+0000","last_peered":"2024-04-18T23:12:25.849949+0000","last_clean":"2024-04-18T23:12:25.849949+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:12:25.849949+0000","last_undegraded":"2024-04-18T23:12:25.849949+0000","last_fullsized":"2024-04-18T23:12:25.849949+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.796699+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.796699+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:12:51.215160+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":194,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.754968+0000","last_change":"2024-04-18T23:12:01.049893+0000","last_active":"2024-04-18T23:12:25.754968+0000","last_peered":"2024-04-18T23:12:25.754968+0000","last_clean":"2024-04-18T23:12:25.754968+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:12:25.754968+0000","last_undegraded":"2024-04-18T23:12:25.754968+0000","last_fullsized":"2024-04-18T23:12:25.754968+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:01.049599+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:01.049599+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:09.776200+0000","scrub_duration":403,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":187,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.956577+0000","last_change":"2024-04-18T23:10:45.591981+0000","last_active":"2024-04-18T23:12:25.956577+0000","last_peered":"2024-04-18T23:12:25.956577+0000","last_clean":"2024-04-18T23:12:25.956577+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:12:25.956577+0000","last_undegraded":"2024-04-18T23:12:25.956577+0000","last_fullsized":"2024-04-18T23:12:25.956577+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:10:44.375476+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":194,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.754888+0000","last_change":"2024-04-18T23:12:14.068230+0000","last_active":"2024-04-18T23:12:25.754888+0000","last_peered":"2024-04-18T23:12:25.754888+0000","last_clean":"2024-04-18T23:12:25.754888+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:12:25.754888+0000","last_undegraded":"2024-04-18T23:12:25.754888+0000","last_fullsized":"2024-04-18T23:12:25.754888+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:14.068006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:14.068006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.692265+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":194,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:26.015487+0000","last_change":"2024-04-18T23:11:47.562908+0000","last_active":"2024-04-18T23:12:26.015487+0000","last_peered":"2024-04-18T23:12:26.015487+0000","last_clean":"2024-04-18T23:12:26.015487+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:12:26.015487+0000","last_undegraded":"2024-04-18T23:12:26.015487+0000","last_fullsized":"2024-04-18T23:12:26.015487+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:11:47.562687+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:11:47.562687+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:01.793032+0000","scrub_duration":387,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":195,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.955497+0000","last_change":"2024-04-18T23:12:13.697974+0000","last_active":"2024-04-18T23:12:25.955497+0000","last_peered":"2024-04-18T23:12:25.955497+0000","last_clean":"2024-04-18T23:12:25.955497+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:12:25.955497+0000","last_undegraded":"2024-04-18T23:12:25.955497+0000","last_fullsized":"2024-04-18T23:12:25.955497+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.697732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.697732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:38.678587+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":194,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.753532+0000","last_change":"2024-04-18T23:12:15.125832+0000","last_active":"2024-04-18T23:12:25.753532+0000","last_peered":"2024-04-18T23:12:25.753532+0000","last_clean":"2024-04-18T23:12:25.753532+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:12:25.753532+0000","last_undegraded":"2024-04-18T23:12:25.753532+0000","last_fullsized":"2024-04-18T23:12:25.753532+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_clean_scrub_stamp":"2024-04-18T23:12:15.125684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.727019+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":115,"state":"active+clean","last_fresh":"2024-04-18T23:12:25.756438+0000","last_change":"2024-04-18T23:12:13.759380+0000","last_active":"2024-04-18T23:12:25.756438+0000","last_peered":"2024-04-18T23:12:25.756438+0000","last_clean":"2024-04-18T23:12:25.756438+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:12:25.756438+0000","last_undegraded":"2024-04-18T23:12:25.756438+0000","last_fullsized":"2024-04-18T23:12:25.756438+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.758998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.758998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:41.357877+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808844,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30244,"kb_used_data":1688,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94341596,"statfs":{"total":96636764160,"available":96605794304,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280537,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1545488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313240,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313240,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32836,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1695744,"data_stored":997572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313241,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,6,7,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313241,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7,8,9,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313241,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32840,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339000,"statfs":{"total":96636764160,"available":96603136000,"internally_reserved":0,"allocated":1695744,"data_stored":997572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313239,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32876,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338964,"statfs":{"total":96636764160,"available":96603099136,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313239,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32836,"kb_used_data":1656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339004,"statfs":{"total":96636764160,"available":96603140096,"internally_reserved":0,"allocated":1695744,"data_stored":997572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313240,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32880,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338960,"statfs":{"total":96636764160,"available":96603095040,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280537,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33392,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338448,"statfs":{"total":96636764160,"available":96602570752,"internally_reserved":0,"allocated":2260992,"data_stored":1545488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280537,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50928,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320912,"statfs":{"total":96636764160,"available":96584613888,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280538,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32868,"kb_used_data":1688,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338972,"statfs":{"total":96636764160,"available":96603107328,"internally_reserved":0,"allocated":1728512,"data_stored":1016560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280537,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":32888,"kb_used_data":1704,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338952,"statfs":{"total":96636764160,"available":96603086848,"internally_reserved":0,"allocated":1744896,"data_stored":1026054,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:12:31.432 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-18T23:12:31.432 INFO:tasks.thrashosds.thrasher:choose_action: min_in 8 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-18T23:12:31.432 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-04-18T23:12:31.433 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-04-18T23:12:31.598 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-04-18T23:12:31.759 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-04-18T23:12:31.927 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-04-18T23:12:32.087 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-04-18T23:12:32.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-04-18T23:12:32.417 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-04-18T23:12:32.580 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-04-18T23:12:32.738 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 reset_purged_snaps_last 2024-04-18T23:12:32.893 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 reset_purged_snaps_last 2024-04-18T23:12:33.054 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 reset_purged_snaps_last 2024-04-18T23:12:33.202 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 reset_purged_snaps_last 2024-04-18T23:12:33.361 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 reset_purged_snaps_last 2024-04-18T23:12:33.508 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 reset_purged_snaps_last 2024-04-18T23:12:33.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 reset_purged_snaps_last 2024-04-18T23:12:33.801 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 reset_purged_snaps_last 2024-04-18T23:12:38.950 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-18T23:12:39.101 INFO:teuthology.orchestra.run.smithi073.stdout:mon.a: {} 2024-04-18T23:12:39.101 INFO:teuthology.orchestra.run.smithi073.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T23:12:39.114 INFO:teuthology.orchestra.run.smithi073.stdout:mon.b: {} 2024-04-18T23:12:39.114 INFO:teuthology.orchestra.run.smithi073.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T23:12:39.128 INFO:teuthology.orchestra.run.smithi073.stdout:mon.c: {} 2024-04-18T23:12:39.128 INFO:teuthology.orchestra.run.smithi073.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-18T23:12:39.144 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-18T23:12:40.383 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.0 to 1 (10000) 2024-04-18T23:12:40.405 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-18T23:12:41.393 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-18T23:12:41.421 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-18T23:12:42.395 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.4 to 1 (10000) 2024-04-18T23:12:42.419 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-04-18T23:12:43.413 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-04-18T23:12:43.435 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 8 1 2024-04-18T23:12:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.8 to 1 (10000) 2024-04-18T23:12:44.441 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 8 1 2024-04-18T23:12:45.427 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.8 primary-affinity to 1 (8655362) 2024-04-18T23:12:45.454 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 12 1 2024-04-18T23:12:46.436 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.12 to 1 (10000) 2024-04-18T23:12:46.459 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 12 1 2024-04-18T23:12:47.446 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.12 primary-affinity to 1 (8655362) 2024-04-18T23:12:47.471 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-18T23:12:48.455 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.1 to 1 (10000) 2024-04-18T23:12:48.479 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-18T23:12:49.464 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-18T23:12:49.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-18T23:12:50.484 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.5 to 1 (10000) 2024-04-18T23:12:50.506 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-04-18T23:12:51.485 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-04-18T23:12:51.511 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 9 1 2024-04-18T23:12:52.495 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.9 to 1 (10000) 2024-04-18T23:12:52.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 9 1 2024-04-18T23:12:53.504 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.9 primary-affinity to 1 (8655362) 2024-04-18T23:12:53.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 13 1 2024-04-18T23:12:54.514 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.13 to 1 (10000) 2024-04-18T23:12:54.539 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 13 1 2024-04-18T23:12:55.528 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.13 primary-affinity to 1 (8655362) 2024-04-18T23:12:55.553 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-18T23:12:56.531 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.2 to 1 (10000) 2024-04-18T23:12:56.557 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-18T23:12:57.541 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-18T23:12:57.567 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-18T23:12:58.550 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.6 to 1 (10000) 2024-04-18T23:12:58.576 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-04-18T23:12:59.563 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-04-18T23:12:59.586 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 10 1 2024-04-18T23:13:00.575 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.10 to 1 (10000) 2024-04-18T23:13:00.602 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 10 1 2024-04-18T23:13:01.589 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.10 primary-affinity to 1 (8655362) 2024-04-18T23:13:01.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 14 1 2024-04-18T23:13:02.593 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.14 to 1 (10000) 2024-04-18T23:13:02.621 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 14 1 2024-04-18T23:13:03.608 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.14 primary-affinity to 1 (8655362) 2024-04-18T23:13:03.630 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-18T23:13:04.612 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.3 to 1 (10000) 2024-04-18T23:13:04.640 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-18T23:13:05.631 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-18T23:13:05.655 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-18T23:13:06.632 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.7 to 1 (10000) 2024-04-18T23:13:06.654 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-18T23:13:07.643 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-18T23:13:07.665 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 11 1 2024-04-18T23:13:08.643 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.11 to 1 (10000) 2024-04-18T23:13:08.668 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 11 1 2024-04-18T23:13:09.661 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.11 primary-affinity to 1 (8655362) 2024-04-18T23:13:09.685 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 15 1 2024-04-18T23:13:10.665 INFO:teuthology.orchestra.run.smithi073.stderr:reweighted osd.15 to 1 (10000) 2024-04-18T23:13:10.691 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 15 1 2024-04-18T23:13:11.684 INFO:teuthology.orchestra.run.smithi073.stderr:set osd.15 primary-affinity to 1 (8655362) 2024-04-18T23:13:11.713 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-18T23:13:11.713 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-18T23:13:11.713 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-18T23:13:11.713 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-18T23:13:11.713 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-18T23:13:11.714 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T23:13:12.001 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:12.002 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":152,"fsid":"b8d0ffd9-6668-442b-a039-a24cea734f3e","created":"2024-04-18T23:10:28.548637+0000","modified":"2024-04-18T23:13:11.670744+0000","last_up_change":"2024-04-18T23:11:32.525078+0000","last_in_change":"2024-04-18T23:10:33.002063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T23:10:41.349753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T23:10:43.462157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3af51a22-0d99-447f-8e09-ef2a2077dcba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6803","nonce":169883533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6805","nonce":169883533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6817","nonce":169883533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6808","nonce":169883533}]},"public_addr":"172.21.15.73:6803/169883533","cluster_addr":"172.21.15.73:6805/169883533","heartbeat_back_addr":"172.21.15.73:6817/169883533","heartbeat_front_addr":"172.21.15.73:6808/169883533","state":["exists","up"]},{"osd":1,"uuid":"f50deb0a-c98a-45ec-9ee3-be20aff17eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6809","nonce":3147085605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6811","nonce":3147085605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6816","nonce":3147085605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6813","nonce":3147085605}]},"public_addr":"172.21.15.120:6809/3147085605","cluster_addr":"172.21.15.120:6811/3147085605","heartbeat_back_addr":"172.21.15.120:6816/3147085605","heartbeat_front_addr":"172.21.15.120:6813/3147085605","state":["exists","up"]},{"osd":2,"uuid":"b6556b6f-9ef8-462d-ae9f-b59713dec207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6809","nonce":1984461220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6819","nonce":1984461220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6831","nonce":1984461220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6823","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6827","nonce":1984461220}]},"public_addr":"172.21.15.136:6809/1984461220","cluster_addr":"172.21.15.136:6819/1984461220","heartbeat_back_addr":"172.21.15.136:6831/1984461220","heartbeat_front_addr":"172.21.15.136:6827/1984461220","state":["exists","up"]},{"osd":3,"uuid":"2448b4d0-af77-4951-bcf3-f2752e7d13b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6817","nonce":2108797520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6819","nonce":2108797520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6823","nonce":2108797520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6821","nonce":2108797520}]},"public_addr":"172.21.15.142:6817/2108797520","cluster_addr":"172.21.15.142:6819/2108797520","heartbeat_back_addr":"172.21.15.142:6823/2108797520","heartbeat_front_addr":"172.21.15.142:6821/2108797520","state":["exists","up"]},{"osd":4,"uuid":"d8ae2413-5474-40ca-b256-04b012595463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6812","nonce":1768990367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6820","nonce":1768990367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6831","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6833","nonce":1768990367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6823","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6827","nonce":1768990367}]},"public_addr":"172.21.15.73:6812/1768990367","cluster_addr":"172.21.15.73:6820/1768990367","heartbeat_back_addr":"172.21.15.73:6833/1768990367","heartbeat_front_addr":"172.21.15.73:6827/1768990367","state":["exists","up"]},{"osd":5,"uuid":"3515f9b0-470c-4db3-b456-a3985a36bf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6801","nonce":4018604021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6803","nonce":4018604021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6807","nonce":4018604021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6805","nonce":4018604021}]},"public_addr":"172.21.15.120:6801/4018604021","cluster_addr":"172.21.15.120:6803/4018604021","heartbeat_back_addr":"172.21.15.120:6807/4018604021","heartbeat_front_addr":"172.21.15.120:6805/4018604021","state":["exists","up"]},{"osd":6,"uuid":"12d2f5e8-9eaa-4c21-a250-1db8c4e641cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6803","nonce":2261503449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6811","nonce":2261503449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6826","nonce":2261503449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6818","nonce":2261503449}]},"public_addr":"172.21.15.136:6803/2261503449","cluster_addr":"172.21.15.136:6811/2261503449","heartbeat_back_addr":"172.21.15.136:6826/2261503449","heartbeat_front_addr":"172.21.15.136:6818/2261503449","state":["exists","up"]},{"osd":7,"uuid":"c9e5e983-106f-4d01-a153-cdf7d656a091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6804","nonce":1398734507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6807","nonce":1398734507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6814","nonce":1398734507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6810","nonce":1398734507}]},"public_addr":"172.21.15.142:6804/1398734507","cluster_addr":"172.21.15.142:6807/1398734507","heartbeat_back_addr":"172.21.15.142:6814/1398734507","heartbeat_front_addr":"172.21.15.142:6810/1398734507","state":["exists","up"]},{"osd":8,"uuid":"a711d0fa-f5a7-4146-b280-8ee969afbb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6815","nonce":786826444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6819","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6821","nonce":786826444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6829","nonce":786826444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6826","nonce":786826444}]},"public_addr":"172.21.15.73:6815/786826444","cluster_addr":"172.21.15.73:6821/786826444","heartbeat_back_addr":"172.21.15.73:6829/786826444","heartbeat_front_addr":"172.21.15.73:6826/786826444","state":["exists","up"]},{"osd":9,"uuid":"914af14e-f90d-42de-918a-d925f50ccfb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6817","nonce":2287493993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6819","nonce":2287493993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6823","nonce":2287493993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6821","nonce":2287493993}]},"public_addr":"172.21.15.120:6817/2287493993","cluster_addr":"172.21.15.120:6819/2287493993","heartbeat_back_addr":"172.21.15.120:6823/2287493993","heartbeat_front_addr":"172.21.15.120:6821/2287493993","state":["exists","up"]},{"osd":10,"uuid":"6ad314a3-6c86-46fd-b1a6-d20911da6d24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6802","nonce":1419708480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6808","nonce":1419708480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6825","nonce":1419708480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6817","nonce":1419708480}]},"public_addr":"172.21.15.136:6802/1419708480","cluster_addr":"172.21.15.136:6808/1419708480","heartbeat_back_addr":"172.21.15.136:6825/1419708480","heartbeat_front_addr":"172.21.15.136:6817/1419708480","state":["exists","up"]},{"osd":11,"uuid":"2ea8b70d-e122-4c29-8742-18d341fd8668","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6825","nonce":2871475141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6827","nonce":2871475141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6831","nonce":2871475141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6829","nonce":2871475141}]},"public_addr":"172.21.15.142:6825/2871475141","cluster_addr":"172.21.15.142:6827/2871475141","heartbeat_back_addr":"172.21.15.142:6831/2871475141","heartbeat_front_addr":"172.21.15.142:6829/2871475141","state":["exists","up"]},{"osd":12,"uuid":"24086ac0-a92d-400b-9b3d-f48992dc8679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6810","nonce":3885787537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6818","nonce":3885787537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6832","nonce":3885787537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6825","nonce":3885787537}]},"public_addr":"172.21.15.73:6810/3885787537","cluster_addr":"172.21.15.73:6818/3885787537","heartbeat_back_addr":"172.21.15.73:6832/3885787537","heartbeat_front_addr":"172.21.15.73:6825/3885787537","state":["exists","up"]},{"osd":13,"uuid":"da98eea5-e373-4325-adda-75f747ad6a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6825","nonce":2634549424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6827","nonce":2634549424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6831","nonce":2634549424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6829","nonce":2634549424}]},"public_addr":"172.21.15.120:6825/2634549424","cluster_addr":"172.21.15.120:6827/2634549424","heartbeat_back_addr":"172.21.15.120:6831/2634549424","heartbeat_front_addr":"172.21.15.120:6829/2634549424","state":["exists","up"]},{"osd":14,"uuid":"d03df2d4-8d1c-4c2a-9719-d64ad780acc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6810","nonce":381469963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6816","nonce":381469963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6829","nonce":381469963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6824","nonce":381469963}]},"public_addr":"172.21.15.136:6810/381469963","cluster_addr":"172.21.15.136:6816/381469963","heartbeat_back_addr":"172.21.15.136:6829/381469963","heartbeat_front_addr":"172.21.15.136:6824/381469963","state":["exists","up"]},{"osd":15,"uuid":"7aa92378-d336-41be-85f1-2eb582f54947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":22,"last_clean_end":48,"up_from":67,"up_thru":72,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6801","nonce":1087448681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6806","nonce":1087448681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6815","nonce":1087448681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6811","nonce":1087448681}]},"public_addr":"172.21.15.142:6801/1087448681","cluster_addr":"172.21.15.142:6806/1087448681","heartbeat_back_addr":"172.21.15.142:6815/1087448681","heartbeat_front_addr":"172.21.15.142:6811/1087448681","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:10.978166+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:05.781059+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:07.537155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:58.962904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:06.855916+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:51.333967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:06.821551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:05.573559+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:59.025889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:01.307602+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:09.979832+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:00.497097+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:59.226387+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:02.057520+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:40.665511+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:02.044564+0000","dead_epoch":49}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T23:13:12.019 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-18T23:13:12.019 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T23:13:12.308 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:12.308 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":152,"fsid":"b8d0ffd9-6668-442b-a039-a24cea734f3e","created":"2024-04-18T23:10:28.548637+0000","modified":"2024-04-18T23:13:11.670744+0000","last_up_change":"2024-04-18T23:11:32.525078+0000","last_in_change":"2024-04-18T23:10:33.002063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T23:10:41.349753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T23:10:43.462157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3af51a22-0d99-447f-8e09-ef2a2077dcba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6803","nonce":169883533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6805","nonce":169883533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6817","nonce":169883533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6808","nonce":169883533}]},"public_addr":"172.21.15.73:6803/169883533","cluster_addr":"172.21.15.73:6805/169883533","heartbeat_back_addr":"172.21.15.73:6817/169883533","heartbeat_front_addr":"172.21.15.73:6808/169883533","state":["exists","up"]},{"osd":1,"uuid":"f50deb0a-c98a-45ec-9ee3-be20aff17eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6809","nonce":3147085605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6811","nonce":3147085605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6816","nonce":3147085605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6813","nonce":3147085605}]},"public_addr":"172.21.15.120:6809/3147085605","cluster_addr":"172.21.15.120:6811/3147085605","heartbeat_back_addr":"172.21.15.120:6816/3147085605","heartbeat_front_addr":"172.21.15.120:6813/3147085605","state":["exists","up"]},{"osd":2,"uuid":"b6556b6f-9ef8-462d-ae9f-b59713dec207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6809","nonce":1984461220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6819","nonce":1984461220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6831","nonce":1984461220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6823","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6827","nonce":1984461220}]},"public_addr":"172.21.15.136:6809/1984461220","cluster_addr":"172.21.15.136:6819/1984461220","heartbeat_back_addr":"172.21.15.136:6831/1984461220","heartbeat_front_addr":"172.21.15.136:6827/1984461220","state":["exists","up"]},{"osd":3,"uuid":"2448b4d0-af77-4951-bcf3-f2752e7d13b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6817","nonce":2108797520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6819","nonce":2108797520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6823","nonce":2108797520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6821","nonce":2108797520}]},"public_addr":"172.21.15.142:6817/2108797520","cluster_addr":"172.21.15.142:6819/2108797520","heartbeat_back_addr":"172.21.15.142:6823/2108797520","heartbeat_front_addr":"172.21.15.142:6821/2108797520","state":["exists","up"]},{"osd":4,"uuid":"d8ae2413-5474-40ca-b256-04b012595463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6812","nonce":1768990367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6820","nonce":1768990367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6831","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6833","nonce":1768990367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6823","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6827","nonce":1768990367}]},"public_addr":"172.21.15.73:6812/1768990367","cluster_addr":"172.21.15.73:6820/1768990367","heartbeat_back_addr":"172.21.15.73:6833/1768990367","heartbeat_front_addr":"172.21.15.73:6827/1768990367","state":["exists","up"]},{"osd":5,"uuid":"3515f9b0-470c-4db3-b456-a3985a36bf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6801","nonce":4018604021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6803","nonce":4018604021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6807","nonce":4018604021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6805","nonce":4018604021}]},"public_addr":"172.21.15.120:6801/4018604021","cluster_addr":"172.21.15.120:6803/4018604021","heartbeat_back_addr":"172.21.15.120:6807/4018604021","heartbeat_front_addr":"172.21.15.120:6805/4018604021","state":["exists","up"]},{"osd":6,"uuid":"12d2f5e8-9eaa-4c21-a250-1db8c4e641cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6803","nonce":2261503449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6811","nonce":2261503449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6826","nonce":2261503449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6818","nonce":2261503449}]},"public_addr":"172.21.15.136:6803/2261503449","cluster_addr":"172.21.15.136:6811/2261503449","heartbeat_back_addr":"172.21.15.136:6826/2261503449","heartbeat_front_addr":"172.21.15.136:6818/2261503449","state":["exists","up"]},{"osd":7,"uuid":"c9e5e983-106f-4d01-a153-cdf7d656a091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6804","nonce":1398734507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6807","nonce":1398734507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6814","nonce":1398734507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6810","nonce":1398734507}]},"public_addr":"172.21.15.142:6804/1398734507","cluster_addr":"172.21.15.142:6807/1398734507","heartbeat_back_addr":"172.21.15.142:6814/1398734507","heartbeat_front_addr":"172.21.15.142:6810/1398734507","state":["exists","up"]},{"osd":8,"uuid":"a711d0fa-f5a7-4146-b280-8ee969afbb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6815","nonce":786826444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6819","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6821","nonce":786826444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6829","nonce":786826444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6826","nonce":786826444}]},"public_addr":"172.21.15.73:6815/786826444","cluster_addr":"172.21.15.73:6821/786826444","heartbeat_back_addr":"172.21.15.73:6829/786826444","heartbeat_front_addr":"172.21.15.73:6826/786826444","state":["exists","up"]},{"osd":9,"uuid":"914af14e-f90d-42de-918a-d925f50ccfb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6817","nonce":2287493993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6819","nonce":2287493993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6823","nonce":2287493993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6821","nonce":2287493993}]},"public_addr":"172.21.15.120:6817/2287493993","cluster_addr":"172.21.15.120:6819/2287493993","heartbeat_back_addr":"172.21.15.120:6823/2287493993","heartbeat_front_addr":"172.21.15.120:6821/2287493993","state":["exists","up"]},{"osd":10,"uuid":"6ad314a3-6c86-46fd-b1a6-d20911da6d24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6802","nonce":1419708480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6808","nonce":1419708480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6825","nonce":1419708480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6817","nonce":1419708480}]},"public_addr":"172.21.15.136:6802/1419708480","cluster_addr":"172.21.15.136:6808/1419708480","heartbeat_back_addr":"172.21.15.136:6825/1419708480","heartbeat_front_addr":"172.21.15.136:6817/1419708480","state":["exists","up"]},{"osd":11,"uuid":"2ea8b70d-e122-4c29-8742-18d341fd8668","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6825","nonce":2871475141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6827","nonce":2871475141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6831","nonce":2871475141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6829","nonce":2871475141}]},"public_addr":"172.21.15.142:6825/2871475141","cluster_addr":"172.21.15.142:6827/2871475141","heartbeat_back_addr":"172.21.15.142:6831/2871475141","heartbeat_front_addr":"172.21.15.142:6829/2871475141","state":["exists","up"]},{"osd":12,"uuid":"24086ac0-a92d-400b-9b3d-f48992dc8679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6810","nonce":3885787537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6818","nonce":3885787537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6832","nonce":3885787537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6825","nonce":3885787537}]},"public_addr":"172.21.15.73:6810/3885787537","cluster_addr":"172.21.15.73:6818/3885787537","heartbeat_back_addr":"172.21.15.73:6832/3885787537","heartbeat_front_addr":"172.21.15.73:6825/3885787537","state":["exists","up"]},{"osd":13,"uuid":"da98eea5-e373-4325-adda-75f747ad6a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6825","nonce":2634549424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6827","nonce":2634549424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6831","nonce":2634549424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6829","nonce":2634549424}]},"public_addr":"172.21.15.120:6825/2634549424","cluster_addr":"172.21.15.120:6827/2634549424","heartbeat_back_addr":"172.21.15.120:6831/2634549424","heartbeat_front_addr":"172.21.15.120:6829/2634549424","state":["exists","up"]},{"osd":14,"uuid":"d03df2d4-8d1c-4c2a-9719-d64ad780acc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6810","nonce":381469963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6816","nonce":381469963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6829","nonce":381469963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6824","nonce":381469963}]},"public_addr":"172.21.15.136:6810/381469963","cluster_addr":"172.21.15.136:6816/381469963","heartbeat_back_addr":"172.21.15.136:6829/381469963","heartbeat_front_addr":"172.21.15.136:6824/381469963","state":["exists","up"]},{"osd":15,"uuid":"7aa92378-d336-41be-85f1-2eb582f54947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":22,"last_clean_end":48,"up_from":67,"up_thru":72,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6801","nonce":1087448681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6806","nonce":1087448681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6815","nonce":1087448681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6811","nonce":1087448681}]},"public_addr":"172.21.15.142:6801/1087448681","cluster_addr":"172.21.15.142:6806/1087448681","heartbeat_back_addr":"172.21.15.142:6815/1087448681","heartbeat_front_addr":"172.21.15.142:6811/1087448681","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:10.978166+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:05.781059+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:07.537155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:58.962904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:06.855916+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:51.333967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:06.821551+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:05.573559+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:59.025889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:01.307602+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:09.979832+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:00.497097+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:59.226387+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:02.057520+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:40.665511+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:02.044564+0000","dead_epoch":49}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T23:13:12.325 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-18T23:13:12.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-18T23:13:12.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-18T23:13:12.327 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-18T23:13:12.327 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-18T23:13:12.328 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-18T23:13:12.328 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-18T23:13:12.328 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-18T23:13:12.329 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 flush_pg_stats 2024-04-18T23:13:12.329 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 flush_pg_stats 2024-04-18T23:13:12.329 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 flush_pg_stats 2024-04-18T23:13:12.330 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 flush_pg_stats 2024-04-18T23:13:12.330 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 flush_pg_stats 2024-04-18T23:13:12.331 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 flush_pg_stats 2024-04-18T23:13:12.331 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 flush_pg_stats 2024-04-18T23:13:12.331 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 flush_pg_stats 2024-04-18T23:13:12.718 INFO:teuthology.orchestra.run.smithi073.stdout:90194313251 2024-04-18T23:13:12.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-18T23:13:12.780 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:12.780 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.14 2024-04-18T23:13:12.792 INFO:teuthology.orchestra.run.smithi073.stdout:90194313249 2024-04-18T23:13:12.793 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.11 2024-04-18T23:13:12.801 INFO:teuthology.orchestra.run.smithi073.stdout:94489280548 2024-04-18T23:13:12.803 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-18T23:13:12.804 INFO:teuthology.orchestra.run.smithi073.stdout:90194313251 2024-04-18T23:13:12.804 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-18T23:13:12.825 INFO:teuthology.orchestra.run.smithi073.stdout:90194313250 2024-04-18T23:13:12.825 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-18T23:13:12.854 INFO:teuthology.orchestra.run.smithi073.stdout:90194313250 2024-04-18T23:13:12.855 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-18T23:13:12.858 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:12.858 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.10 2024-04-18T23:13:12.859 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:12.859 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-18T23:13:12.876 INFO:teuthology.orchestra.run.smithi073.stdout:287762808854 2024-04-18T23:13:12.877 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.15 2024-04-18T23:13:12.900 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:12.900 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.13 2024-04-18T23:13:12.903 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:12.903 INFO:teuthology.orchestra.run.smithi073.stdout:90194313250 2024-04-18T23:13:12.904 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.9 2024-04-18T23:13:12.904 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.12 2024-04-18T23:13:12.914 INFO:teuthology.orchestra.run.smithi073.stdout:90194313251 2024-04-18T23:13:12.915 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-18T23:13:12.934 INFO:teuthology.orchestra.run.smithi073.stdout:90194313249 2024-04-18T23:13:12.934 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.8 2024-04-18T23:13:12.964 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:12.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-18T23:13:13.748 INFO:teuthology.orchestra.run.smithi073.stdout:287762808854 2024-04-18T23:13:13.779 INFO:teuthology.orchestra.run.smithi073.stdout:90194313251 2024-04-18T23:13:13.797 INFO:teuthology.orchestra.run.smithi073.stdout:90194313251 2024-04-18T23:13:13.816 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:13.822 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313251 got 90194313251 for osd.3 2024-04-18T23:13:13.822 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.828 INFO:tasks.ceph.ceph_manager.ceph:need seq 287762808854 got 287762808854 for osd.15 2024-04-18T23:13:13.828 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.839 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:13.839 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:13.845 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313251 got 90194313251 for osd.5 2024-04-18T23:13:13.845 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.863 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280547 got 94489280547 for osd.14 2024-04-18T23:13:13.864 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.872 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:13.872 INFO:teuthology.orchestra.run.smithi073.stdout:90194313249 2024-04-18T23:13:13.874 INFO:teuthology.orchestra.run.smithi073.stdout:94489280548 2024-04-18T23:13:13.882 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280547 got 94489280547 for osd.10 2024-04-18T23:13:13.882 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.906 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280547 got 94489280547 for osd.13 2024-04-18T23:13:13.906 INFO:teuthology.orchestra.run.smithi073.stdout:90194313250 2024-04-18T23:13:13.906 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.909 INFO:teuthology.orchestra.run.smithi073.stdout:90194313251 2024-04-18T23:13:13.910 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313249 got 90194313249 for osd.11 2024-04-18T23:13:13.910 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.911 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280547 got 94489280547 for osd.2 2024-04-18T23:13:13.911 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.916 INFO:teuthology.orchestra.run.smithi073.stdout:90194313250 2024-04-18T23:13:13.924 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280548 got 94489280548 for osd.7 2024-04-18T23:13:13.924 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.928 INFO:teuthology.orchestra.run.smithi073.stdout:90194313250 2024-04-18T23:13:13.941 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313251 got 90194313251 for osd.4 2024-04-18T23:13:13.941 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.942 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313250 got 90194313250 for osd.12 2024-04-18T23:13:13.942 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.946 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313250 got 90194313250 for osd.0 2024-04-18T23:13:13.947 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.957 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313250 got 90194313250 for osd.1 2024-04-18T23:13:13.957 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.970 INFO:teuthology.orchestra.run.smithi073.stdout:90194313249 2024-04-18T23:13:13.978 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:13.978 INFO:teuthology.orchestra.run.smithi073.stdout:94489280547 2024-04-18T23:13:13.991 INFO:tasks.ceph.ceph_manager.ceph:need seq 90194313249 got 90194313249 for osd.8 2024-04-18T23:13:13.991 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:13.999 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280547 got 94489280547 for osd.9 2024-04-18T23:13:14.000 INFO:tasks.ceph.ceph_manager.ceph:need seq 94489280547 got 94489280547 for osd.6 2024-04-18T23:13:14.000 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:14.000 DEBUG:teuthology.parallel:result is None 2024-04-18T23:13:14.000 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-18T23:13:14.000 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:13:14.313 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:14.314 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:13:14.330 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-04-18T23:13:13.699771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":551728,"kb_used_data":37376,"kb_used_omap":35,"kb_used_meta":514076,"kb_avail":1509397712,"statfs":{"total":1546188226560,"available":1545623257088,"internally_reserved":0,"allocated":38273024,"data_stored":22894936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":526414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.041336"},"pg_stats":[{"pgid":"1.0","version":"33'157","reported_seq":474,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.837928+0000","last_change":"2024-04-18T23:13:05.784628+0000","last_active":"2024-04-18T23:13:12.837928+0000","last_peered":"2024-04-18T23:13:12.837928+0000","last_clean":"2024-04-18T23:13:12.837928+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:13:12.837928+0000","last_undegraded":"2024-04-18T23:13:12.837928+0000","last_fullsized":"2024-04-18T23:13:12.837928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:13:05.784378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:13:05.784378+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.622259+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":276,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.684445+0000","last_change":"2024-04-18T23:12:53.633755+0000","last_active":"2024-04-18T23:13:11.684445+0000","last_peered":"2024-04-18T23:13:11.684445+0000","last_clean":"2024-04-18T23:13:11.684445+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:13:11.684445+0000","last_undegraded":"2024-04-18T23:13:11.684445+0000","last_fullsized":"2024-04-18T23:13:11.684445+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:53.633521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:53.633521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:23.488690+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":269,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.883376+0000","last_change":"2024-04-18T23:13:11.072187+0000","last_active":"2024-04-18T23:13:11.883376+0000","last_peered":"2024-04-18T23:13:11.883376+0000","last_clean":"2024-04-18T23:13:11.883376+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:13:11.883376+0000","last_undegraded":"2024-04-18T23:13:11.883376+0000","last_fullsized":"2024-04-18T23:13:11.883376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:11.071951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:11.071951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:25.724133+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.737895+0000","last_change":"2024-04-18T23:12:29.985926+0000","last_active":"2024-04-18T23:13:12.737895+0000","last_peered":"2024-04-18T23:13:12.737895+0000","last_clean":"2024-04-18T23:13:12.737895+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:13:12.737895+0000","last_undegraded":"2024-04-18T23:13:12.737895+0000","last_fullsized":"2024-04-18T23:13:12.737895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:29.985705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:29.985705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:37.118468+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":261,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.083583+0000","last_change":"2024-04-18T23:12:14.068230+0000","last_active":"2024-04-18T23:13:12.083583+0000","last_peered":"2024-04-18T23:13:12.083583+0000","last_clean":"2024-04-18T23:13:12.083583+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:13:12.083583+0000","last_undegraded":"2024-04-18T23:13:12.083583+0000","last_fullsized":"2024-04-18T23:13:12.083583+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:14.068006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:14.068006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.692265+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":271,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.785810+0000","last_change":"2024-04-18T23:13:03.435843+0000","last_active":"2024-04-18T23:13:12.785810+0000","last_peered":"2024-04-18T23:13:12.785810+0000","last_clean":"2024-04-18T23:13:12.785810+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:13:12.785810+0000","last_undegraded":"2024-04-18T23:13:12.785810+0000","last_fullsized":"2024-04-18T23:13:12.785810+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:03.435600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:03.435600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.618520+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":263,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.562224+0000","last_change":"2024-04-18T23:12:13.697974+0000","last_active":"2024-04-18T23:13:12.562224+0000","last_peered":"2024-04-18T23:13:12.562224+0000","last_clean":"2024-04-18T23:13:12.562224+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:13:12.562224+0000","last_undegraded":"2024-04-18T23:13:12.562224+0000","last_fullsized":"2024-04-18T23:13:12.562224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.697732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.697732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:38.678587+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.559418+0000","last_change":"2024-04-18T23:12:15.125832+0000","last_active":"2024-04-18T23:13:12.559418+0000","last_peered":"2024-04-18T23:13:12.559418+0000","last_clean":"2024-04-18T23:13:12.559418+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:13:12.559418+0000","last_undegraded":"2024-04-18T23:13:12.559418+0000","last_fullsized":"2024-04-18T23:13:12.559418+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_clean_scrub_stamp":"2024-04-18T23:12:15.125684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.727019+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":188,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.847317+0000","last_change":"2024-04-18T23:12:13.759380+0000","last_active":"2024-04-18T23:13:11.847317+0000","last_peered":"2024-04-18T23:13:11.847317+0000","last_clean":"2024-04-18T23:13:11.847317+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:13:11.847317+0000","last_undegraded":"2024-04-18T23:13:11.847317+0000","last_fullsized":"2024-04-18T23:13:11.847317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.758998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.758998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:41.357877+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30840,"kb_used_data":2280,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94341000,"statfs":{"total":96636764160,"available":96605184000,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33972,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337868,"statfs":{"total":96636764160,"available":96601976832,"internally_reserved":0,"allocated":2850816,"data_stored":1889542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33432,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338408,"statfs":{"total":96636764160,"available":96602529792,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,6,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280547,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33980,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337860,"statfs":{"total":96636764160,"available":96601968640,"internally_reserved":0,"allocated":2867200,"data_stored":1899150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51476,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320364,"statfs":{"total":96636764160,"available":96584052736,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33456,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:13:14.331 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:13:14.602 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:14.603 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:13:14.620 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":215,"stamp":"2024-04-18T23:13:13.699771+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":551728,"kb_used_data":37376,"kb_used_omap":35,"kb_used_meta":514076,"kb_avail":1509397712,"statfs":{"total":1546188226560,"available":1545623257088,"internally_reserved":0,"allocated":38273024,"data_stored":22894936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":526414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.041336"},"pg_stats":[{"pgid":"1.0","version":"33'157","reported_seq":474,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.837928+0000","last_change":"2024-04-18T23:13:05.784628+0000","last_active":"2024-04-18T23:13:12.837928+0000","last_peered":"2024-04-18T23:13:12.837928+0000","last_clean":"2024-04-18T23:13:12.837928+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:13:12.837928+0000","last_undegraded":"2024-04-18T23:13:12.837928+0000","last_fullsized":"2024-04-18T23:13:12.837928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:13:05.784378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:13:05.784378+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.622259+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":276,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.684445+0000","last_change":"2024-04-18T23:12:53.633755+0000","last_active":"2024-04-18T23:13:11.684445+0000","last_peered":"2024-04-18T23:13:11.684445+0000","last_clean":"2024-04-18T23:13:11.684445+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:13:11.684445+0000","last_undegraded":"2024-04-18T23:13:11.684445+0000","last_fullsized":"2024-04-18T23:13:11.684445+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:53.633521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:53.633521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:23.488690+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":269,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.883376+0000","last_change":"2024-04-18T23:13:11.072187+0000","last_active":"2024-04-18T23:13:11.883376+0000","last_peered":"2024-04-18T23:13:11.883376+0000","last_clean":"2024-04-18T23:13:11.883376+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:13:11.883376+0000","last_undegraded":"2024-04-18T23:13:11.883376+0000","last_fullsized":"2024-04-18T23:13:11.883376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:11.071951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:11.071951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:25.724133+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.737895+0000","last_change":"2024-04-18T23:12:29.985926+0000","last_active":"2024-04-18T23:13:12.737895+0000","last_peered":"2024-04-18T23:13:12.737895+0000","last_clean":"2024-04-18T23:13:12.737895+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:13:12.737895+0000","last_undegraded":"2024-04-18T23:13:12.737895+0000","last_fullsized":"2024-04-18T23:13:12.737895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:29.985705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:29.985705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:37.118468+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":261,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.083583+0000","last_change":"2024-04-18T23:12:14.068230+0000","last_active":"2024-04-18T23:13:12.083583+0000","last_peered":"2024-04-18T23:13:12.083583+0000","last_clean":"2024-04-18T23:13:12.083583+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:13:12.083583+0000","last_undegraded":"2024-04-18T23:13:12.083583+0000","last_fullsized":"2024-04-18T23:13:12.083583+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:14.068006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:14.068006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.692265+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":271,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.785810+0000","last_change":"2024-04-18T23:13:03.435843+0000","last_active":"2024-04-18T23:13:12.785810+0000","last_peered":"2024-04-18T23:13:12.785810+0000","last_clean":"2024-04-18T23:13:12.785810+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:13:12.785810+0000","last_undegraded":"2024-04-18T23:13:12.785810+0000","last_fullsized":"2024-04-18T23:13:12.785810+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:03.435600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:03.435600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.618520+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":263,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.562224+0000","last_change":"2024-04-18T23:12:13.697974+0000","last_active":"2024-04-18T23:13:12.562224+0000","last_peered":"2024-04-18T23:13:12.562224+0000","last_clean":"2024-04-18T23:13:12.562224+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:13:12.562224+0000","last_undegraded":"2024-04-18T23:13:12.562224+0000","last_fullsized":"2024-04-18T23:13:12.562224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.697732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.697732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:38.678587+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.559418+0000","last_change":"2024-04-18T23:12:15.125832+0000","last_active":"2024-04-18T23:13:12.559418+0000","last_peered":"2024-04-18T23:13:12.559418+0000","last_clean":"2024-04-18T23:13:12.559418+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:13:12.559418+0000","last_undegraded":"2024-04-18T23:13:12.559418+0000","last_fullsized":"2024-04-18T23:13:12.559418+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_clean_scrub_stamp":"2024-04-18T23:12:15.125684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.727019+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":188,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.847317+0000","last_change":"2024-04-18T23:12:13.759380+0000","last_active":"2024-04-18T23:13:11.847317+0000","last_peered":"2024-04-18T23:13:11.847317+0000","last_clean":"2024-04-18T23:13:11.847317+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:13:11.847317+0000","last_undegraded":"2024-04-18T23:13:11.847317+0000","last_fullsized":"2024-04-18T23:13:11.847317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.758998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.758998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:41.357877+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30840,"kb_used_data":2280,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94341000,"statfs":{"total":96636764160,"available":96605184000,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33972,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337868,"statfs":{"total":96636764160,"available":96601976832,"internally_reserved":0,"allocated":2850816,"data_stored":1889542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33432,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338408,"statfs":{"total":96636764160,"available":96602529792,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,6,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280547,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33980,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337860,"statfs":{"total":96636764160,"available":96601968640,"internally_reserved":0,"allocated":2867200,"data_stored":1899150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51476,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320364,"statfs":{"total":96636764160,"available":96584052736,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33456,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:13:14.621 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-18T23:13:14.621 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-18T23:13:14.631 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-18T23:13:14.631 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:13:14.899 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:14.900 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:13:14.916 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-04-18T23:13:14.702227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":551728,"kb_used_data":37376,"kb_used_omap":35,"kb_used_meta":514076,"kb_avail":1509397712,"statfs":{"total":1546188226560,"available":1545623257088,"internally_reserved":0,"allocated":38273024,"data_stored":22894936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":526414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040570"},"pg_stats":[{"pgid":"1.0","version":"33'157","reported_seq":474,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.837928+0000","last_change":"2024-04-18T23:13:05.784628+0000","last_active":"2024-04-18T23:13:12.837928+0000","last_peered":"2024-04-18T23:13:12.837928+0000","last_clean":"2024-04-18T23:13:12.837928+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:13:12.837928+0000","last_undegraded":"2024-04-18T23:13:12.837928+0000","last_fullsized":"2024-04-18T23:13:12.837928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:13:05.784378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:13:05.784378+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.622259+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":276,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.684445+0000","last_change":"2024-04-18T23:12:53.633755+0000","last_active":"2024-04-18T23:13:11.684445+0000","last_peered":"2024-04-18T23:13:11.684445+0000","last_clean":"2024-04-18T23:13:11.684445+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:13:11.684445+0000","last_undegraded":"2024-04-18T23:13:11.684445+0000","last_fullsized":"2024-04-18T23:13:11.684445+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:53.633521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:53.633521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:23.488690+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":269,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.883376+0000","last_change":"2024-04-18T23:13:11.072187+0000","last_active":"2024-04-18T23:13:11.883376+0000","last_peered":"2024-04-18T23:13:11.883376+0000","last_clean":"2024-04-18T23:13:11.883376+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:13:11.883376+0000","last_undegraded":"2024-04-18T23:13:11.883376+0000","last_fullsized":"2024-04-18T23:13:11.883376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:11.071951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:11.071951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:25.724133+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.737895+0000","last_change":"2024-04-18T23:12:29.985926+0000","last_active":"2024-04-18T23:13:12.737895+0000","last_peered":"2024-04-18T23:13:12.737895+0000","last_clean":"2024-04-18T23:13:12.737895+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:13:12.737895+0000","last_undegraded":"2024-04-18T23:13:12.737895+0000","last_fullsized":"2024-04-18T23:13:12.737895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:29.985705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:29.985705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:37.118468+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":261,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.083583+0000","last_change":"2024-04-18T23:12:14.068230+0000","last_active":"2024-04-18T23:13:12.083583+0000","last_peered":"2024-04-18T23:13:12.083583+0000","last_clean":"2024-04-18T23:13:12.083583+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:13:12.083583+0000","last_undegraded":"2024-04-18T23:13:12.083583+0000","last_fullsized":"2024-04-18T23:13:12.083583+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:14.068006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:14.068006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.692265+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":271,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.785810+0000","last_change":"2024-04-18T23:13:03.435843+0000","last_active":"2024-04-18T23:13:12.785810+0000","last_peered":"2024-04-18T23:13:12.785810+0000","last_clean":"2024-04-18T23:13:12.785810+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:13:12.785810+0000","last_undegraded":"2024-04-18T23:13:12.785810+0000","last_fullsized":"2024-04-18T23:13:12.785810+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:03.435600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:03.435600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.618520+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":263,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.562224+0000","last_change":"2024-04-18T23:12:13.697974+0000","last_active":"2024-04-18T23:13:12.562224+0000","last_peered":"2024-04-18T23:13:12.562224+0000","last_clean":"2024-04-18T23:13:12.562224+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:13:12.562224+0000","last_undegraded":"2024-04-18T23:13:12.562224+0000","last_fullsized":"2024-04-18T23:13:12.562224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.697732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.697732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:38.678587+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.559418+0000","last_change":"2024-04-18T23:12:15.125832+0000","last_active":"2024-04-18T23:13:12.559418+0000","last_peered":"2024-04-18T23:13:12.559418+0000","last_clean":"2024-04-18T23:13:12.559418+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:13:12.559418+0000","last_undegraded":"2024-04-18T23:13:12.559418+0000","last_fullsized":"2024-04-18T23:13:12.559418+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_clean_scrub_stamp":"2024-04-18T23:12:15.125684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.727019+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":188,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.847317+0000","last_change":"2024-04-18T23:12:13.759380+0000","last_active":"2024-04-18T23:13:11.847317+0000","last_peered":"2024-04-18T23:13:11.847317+0000","last_clean":"2024-04-18T23:13:11.847317+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:13:11.847317+0000","last_undegraded":"2024-04-18T23:13:11.847317+0000","last_fullsized":"2024-04-18T23:13:11.847317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.758998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.758998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:41.357877+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30840,"kb_used_data":2280,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94341000,"statfs":{"total":96636764160,"available":96605184000,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33972,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337868,"statfs":{"total":96636764160,"available":96601976832,"internally_reserved":0,"allocated":2850816,"data_stored":1889542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33432,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338408,"statfs":{"total":96636764160,"available":96602529792,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,6,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280547,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33980,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337860,"statfs":{"total":96636764160,"available":96601968640,"internally_reserved":0,"allocated":2867200,"data_stored":1899150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51476,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320364,"statfs":{"total":96636764160,"available":96584052736,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33456,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:13:14.918 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:13:15.195 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:15.196 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:13:15.212 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":216,"stamp":"2024-04-18T23:13:14.702227+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":551728,"kb_used_data":37376,"kb_used_omap":35,"kb_used_meta":514076,"kb_avail":1509397712,"statfs":{"total":1546188226560,"available":1545623257088,"internally_reserved":0,"allocated":38273024,"data_stored":22894936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":526414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.040570"},"pg_stats":[{"pgid":"1.0","version":"33'157","reported_seq":474,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.837928+0000","last_change":"2024-04-18T23:13:05.784628+0000","last_active":"2024-04-18T23:13:12.837928+0000","last_peered":"2024-04-18T23:13:12.837928+0000","last_clean":"2024-04-18T23:13:12.837928+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:13:12.837928+0000","last_undegraded":"2024-04-18T23:13:12.837928+0000","last_fullsized":"2024-04-18T23:13:12.837928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:13:05.784378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:13:05.784378+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.622259+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":276,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.684445+0000","last_change":"2024-04-18T23:12:53.633755+0000","last_active":"2024-04-18T23:13:11.684445+0000","last_peered":"2024-04-18T23:13:11.684445+0000","last_clean":"2024-04-18T23:13:11.684445+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:13:11.684445+0000","last_undegraded":"2024-04-18T23:13:11.684445+0000","last_fullsized":"2024-04-18T23:13:11.684445+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:53.633521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:53.633521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:23.488690+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":269,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.883376+0000","last_change":"2024-04-18T23:13:11.072187+0000","last_active":"2024-04-18T23:13:11.883376+0000","last_peered":"2024-04-18T23:13:11.883376+0000","last_clean":"2024-04-18T23:13:11.883376+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:13:11.883376+0000","last_undegraded":"2024-04-18T23:13:11.883376+0000","last_fullsized":"2024-04-18T23:13:11.883376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:11.071951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:11.071951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:25.724133+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.737895+0000","last_change":"2024-04-18T23:12:29.985926+0000","last_active":"2024-04-18T23:13:12.737895+0000","last_peered":"2024-04-18T23:13:12.737895+0000","last_clean":"2024-04-18T23:13:12.737895+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:13:12.737895+0000","last_undegraded":"2024-04-18T23:13:12.737895+0000","last_fullsized":"2024-04-18T23:13:12.737895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:29.985705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:29.985705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:37.118468+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":261,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.083583+0000","last_change":"2024-04-18T23:12:14.068230+0000","last_active":"2024-04-18T23:13:12.083583+0000","last_peered":"2024-04-18T23:13:12.083583+0000","last_clean":"2024-04-18T23:13:12.083583+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:13:12.083583+0000","last_undegraded":"2024-04-18T23:13:12.083583+0000","last_fullsized":"2024-04-18T23:13:12.083583+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:14.068006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:14.068006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.692265+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":271,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.785810+0000","last_change":"2024-04-18T23:13:03.435843+0000","last_active":"2024-04-18T23:13:12.785810+0000","last_peered":"2024-04-18T23:13:12.785810+0000","last_clean":"2024-04-18T23:13:12.785810+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:13:12.785810+0000","last_undegraded":"2024-04-18T23:13:12.785810+0000","last_fullsized":"2024-04-18T23:13:12.785810+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:03.435600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:03.435600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.618520+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":263,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.562224+0000","last_change":"2024-04-18T23:12:13.697974+0000","last_active":"2024-04-18T23:13:12.562224+0000","last_peered":"2024-04-18T23:13:12.562224+0000","last_clean":"2024-04-18T23:13:12.562224+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:13:12.562224+0000","last_undegraded":"2024-04-18T23:13:12.562224+0000","last_fullsized":"2024-04-18T23:13:12.562224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.697732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.697732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:38.678587+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.559418+0000","last_change":"2024-04-18T23:12:15.125832+0000","last_active":"2024-04-18T23:13:12.559418+0000","last_peered":"2024-04-18T23:13:12.559418+0000","last_clean":"2024-04-18T23:13:12.559418+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:13:12.559418+0000","last_undegraded":"2024-04-18T23:13:12.559418+0000","last_fullsized":"2024-04-18T23:13:12.559418+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_clean_scrub_stamp":"2024-04-18T23:12:15.125684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.727019+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":188,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.847317+0000","last_change":"2024-04-18T23:12:13.759380+0000","last_active":"2024-04-18T23:13:11.847317+0000","last_peered":"2024-04-18T23:13:11.847317+0000","last_clean":"2024-04-18T23:13:11.847317+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:13:11.847317+0000","last_undegraded":"2024-04-18T23:13:11.847317+0000","last_fullsized":"2024-04-18T23:13:11.847317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.758998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.758998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:41.357877+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30840,"kb_used_data":2280,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94341000,"statfs":{"total":96636764160,"available":96605184000,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33972,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337868,"statfs":{"total":96636764160,"available":96601976832,"internally_reserved":0,"allocated":2850816,"data_stored":1889542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33432,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338408,"statfs":{"total":96636764160,"available":96602529792,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,6,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280547,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33980,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337860,"statfs":{"total":96636764160,"available":96601968640,"internally_reserved":0,"allocated":2867200,"data_stored":1899150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51476,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320364,"statfs":{"total":96636764160,"available":96584052736,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33456,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:13:15.213 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-18T23:13:15.213 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:13:15.484 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:15.485 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:13:15.501 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":217,"stamp":"2024-04-18T23:13:15.309697+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":551728,"kb_used_data":37376,"kb_used_omap":35,"kb_used_meta":514076,"kb_avail":1509397712,"statfs":{"total":1546188226560,"available":1545623257088,"internally_reserved":0,"allocated":38273024,"data_stored":22894936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":526414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000973"},"pg_stats":[{"pgid":"1.0","version":"33'157","reported_seq":474,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.837928+0000","last_change":"2024-04-18T23:13:05.784628+0000","last_active":"2024-04-18T23:13:12.837928+0000","last_peered":"2024-04-18T23:13:12.837928+0000","last_clean":"2024-04-18T23:13:12.837928+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:13:12.837928+0000","last_undegraded":"2024-04-18T23:13:12.837928+0000","last_fullsized":"2024-04-18T23:13:12.837928+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:13:05.784378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:13:05.784378+0000","objects_scrubbed":2,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.622259+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":276,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.684445+0000","last_change":"2024-04-18T23:12:53.633755+0000","last_active":"2024-04-18T23:13:11.684445+0000","last_peered":"2024-04-18T23:13:11.684445+0000","last_clean":"2024-04-18T23:13:11.684445+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:13:11.684445+0000","last_undegraded":"2024-04-18T23:13:11.684445+0000","last_fullsized":"2024-04-18T23:13:11.684445+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:53.633521+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:53.633521+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:23.488690+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":269,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.883376+0000","last_change":"2024-04-18T23:13:11.072187+0000","last_active":"2024-04-18T23:13:11.883376+0000","last_peered":"2024-04-18T23:13:11.883376+0000","last_clean":"2024-04-18T23:13:11.883376+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:13:11.883376+0000","last_undegraded":"2024-04-18T23:13:11.883376+0000","last_fullsized":"2024-04-18T23:13:11.883376+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:11.071951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:11.071951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:25.724133+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.737895+0000","last_change":"2024-04-18T23:12:29.985926+0000","last_active":"2024-04-18T23:13:12.737895+0000","last_peered":"2024-04-18T23:13:12.737895+0000","last_clean":"2024-04-18T23:13:12.737895+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:13:12.737895+0000","last_undegraded":"2024-04-18T23:13:12.737895+0000","last_fullsized":"2024-04-18T23:13:12.737895+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:29.985705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:29.985705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:37.118468+0000","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":261,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.083583+0000","last_change":"2024-04-18T23:12:14.068230+0000","last_active":"2024-04-18T23:13:12.083583+0000","last_peered":"2024-04-18T23:13:12.083583+0000","last_clean":"2024-04-18T23:13:12.083583+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:13:12.083583+0000","last_undegraded":"2024-04-18T23:13:12.083583+0000","last_fullsized":"2024-04-18T23:13:12.083583+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:14.068006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:14.068006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.692265+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":271,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.785810+0000","last_change":"2024-04-18T23:13:03.435843+0000","last_active":"2024-04-18T23:13:12.785810+0000","last_peered":"2024-04-18T23:13:12.785810+0000","last_clean":"2024-04-18T23:13:12.785810+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:13:12.785810+0000","last_undegraded":"2024-04-18T23:13:12.785810+0000","last_fullsized":"2024-04-18T23:13:12.785810+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:03.435600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:03.435600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:13.618520+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":263,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.562224+0000","last_change":"2024-04-18T23:12:13.697974+0000","last_active":"2024-04-18T23:13:12.562224+0000","last_peered":"2024-04-18T23:13:12.562224+0000","last_clean":"2024-04-18T23:13:12.562224+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:13:12.562224+0000","last_undegraded":"2024-04-18T23:13:12.562224+0000","last_fullsized":"2024-04-18T23:13:12.562224+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.697732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.697732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:38.678587+0000","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":264,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:12.559418+0000","last_change":"2024-04-18T23:12:15.125832+0000","last_active":"2024-04-18T23:13:12.559418+0000","last_peered":"2024-04-18T23:13:12.559418+0000","last_clean":"2024-04-18T23:13:12.559418+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:13:12.559418+0000","last_undegraded":"2024-04-18T23:13:12.559418+0000","last_fullsized":"2024-04-18T23:13:12.559418+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:12:15.125684+0000","last_clean_scrub_stamp":"2024-04-18T23:12:15.125684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.727019+0000","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":188,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-18T23:13:11.847317+0000","last_change":"2024-04-18T23:12:13.759380+0000","last_active":"2024-04-18T23:13:11.847317+0000","last_peered":"2024-04-18T23:13:11.847317+0000","last_clean":"2024-04-18T23:13:11.847317+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:13:11.847317+0000","last_undegraded":"2024-04-18T23:13:11.847317+0000","last_fullsized":"2024-04-18T23:13:11.847317+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.758998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.758998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:41.357877+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808854,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30840,"kb_used_data":2280,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94341000,"statfs":{"total":96636764160,"available":96605184000,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33972,"kb_used_data":2784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337868,"statfs":{"total":96636764160,"available":96601976832,"internally_reserved":0,"allocated":2850816,"data_stored":1889542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313250,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33432,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338408,"statfs":{"total":96636764160,"available":96602529792,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,6,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33452,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338388,"statfs":{"total":96636764160,"available":96602509312,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313251,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313249,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313249,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33460,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338380,"statfs":{"total":96636764160,"available":96602501120,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313250,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280547,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33980,"kb_used_data":2800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337860,"statfs":{"total":96636764160,"available":96601968640,"internally_reserved":0,"allocated":2867200,"data_stored":1899150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280547,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51476,"kb_used_data":2248,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320364,"statfs":{"total":96636764160,"available":96584052736,"internally_reserved":0,"allocated":2301952,"data_stored":1351006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33456,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338384,"statfs":{"total":96636764160,"available":96602505216,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33464,"kb_used_data":2280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338376,"statfs":{"total":96636764160,"available":96602497024,"internally_reserved":0,"allocated":2334720,"data_stored":1370222,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280547,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33444,"kb_used_data":2264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338396,"statfs":{"total":96636764160,"available":96602517504,"internally_reserved":0,"allocated":2318336,"data_stored":1360614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:13:15.503 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-18T23:13:15.776 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:15.776 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":156,"fsid":"b8d0ffd9-6668-442b-a039-a24cea734f3e","created":"2024-04-18T23:10:28.548637+0000","modified":"2024-04-18T23:13:15.712361+0000","last_up_change":"2024-04-18T23:11:32.525078+0000","last_in_change":"2024-04-18T23:10:33.002063+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":16,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-18T23:10:41.349753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"24","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":15.380000114440918,"score_stable":15.380000114440918,"optimal_score":0.12999999523162842,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-18T23:10:43.462157+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"28","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4.0199999809265137,"score_stable":4.0199999809265137,"optimal_score":0.56000000238418579,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3af51a22-0d99-447f-8e09-ef2a2077dcba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6803","nonce":169883533}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6805","nonce":169883533}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6813","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6817","nonce":169883533}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":169883533},{"type":"v1","addr":"172.21.15.73:6808","nonce":169883533}]},"public_addr":"172.21.15.73:6803/169883533","cluster_addr":"172.21.15.73:6805/169883533","heartbeat_back_addr":"172.21.15.73:6817/169883533","heartbeat_front_addr":"172.21.15.73:6808/169883533","state":["exists","up"]},{"osd":1,"uuid":"f50deb0a-c98a-45ec-9ee3-be20aff17eb1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6808","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6809","nonce":3147085605}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6810","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6811","nonce":3147085605}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6814","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6816","nonce":3147085605}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6812","nonce":3147085605},{"type":"v1","addr":"172.21.15.120:6813","nonce":3147085605}]},"public_addr":"172.21.15.120:6809/3147085605","cluster_addr":"172.21.15.120:6811/3147085605","heartbeat_back_addr":"172.21.15.120:6816/3147085605","heartbeat_front_addr":"172.21.15.120:6813/3147085605","state":["exists","up"]},{"osd":2,"uuid":"b6556b6f-9ef8-462d-ae9f-b59713dec207","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6804","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6809","nonce":1984461220}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6815","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6819","nonce":1984461220}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6830","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6831","nonce":1984461220}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6823","nonce":1984461220},{"type":"v1","addr":"172.21.15.136:6827","nonce":1984461220}]},"public_addr":"172.21.15.136:6809/1984461220","cluster_addr":"172.21.15.136:6819/1984461220","heartbeat_back_addr":"172.21.15.136:6831/1984461220","heartbeat_front_addr":"172.21.15.136:6827/1984461220","state":["exists","up"]},{"osd":3,"uuid":"2448b4d0-af77-4951-bcf3-f2752e7d13b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6816","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6817","nonce":2108797520}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6818","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6819","nonce":2108797520}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6822","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6823","nonce":2108797520}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6820","nonce":2108797520},{"type":"v1","addr":"172.21.15.142:6821","nonce":2108797520}]},"public_addr":"172.21.15.142:6817/2108797520","cluster_addr":"172.21.15.142:6819/2108797520","heartbeat_back_addr":"172.21.15.142:6823/2108797520","heartbeat_front_addr":"172.21.15.142:6821/2108797520","state":["exists","up"]},{"osd":4,"uuid":"d8ae2413-5474-40ca-b256-04b012595463","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6809","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6812","nonce":1768990367}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6820","nonce":1768990367}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6831","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6833","nonce":1768990367}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6823","nonce":1768990367},{"type":"v1","addr":"172.21.15.73:6827","nonce":1768990367}]},"public_addr":"172.21.15.73:6812/1768990367","cluster_addr":"172.21.15.73:6820/1768990367","heartbeat_back_addr":"172.21.15.73:6833/1768990367","heartbeat_front_addr":"172.21.15.73:6827/1768990367","state":["exists","up"]},{"osd":5,"uuid":"3515f9b0-470c-4db3-b456-a3985a36bf1f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":70,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6800","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6801","nonce":4018604021}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6802","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6803","nonce":4018604021}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6806","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6807","nonce":4018604021}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6804","nonce":4018604021},{"type":"v1","addr":"172.21.15.120:6805","nonce":4018604021}]},"public_addr":"172.21.15.120:6801/4018604021","cluster_addr":"172.21.15.120:6803/4018604021","heartbeat_back_addr":"172.21.15.120:6807/4018604021","heartbeat_front_addr":"172.21.15.120:6805/4018604021","state":["exists","up"]},{"osd":6,"uuid":"12d2f5e8-9eaa-4c21-a250-1db8c4e641cc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6801","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6803","nonce":2261503449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6807","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6811","nonce":2261503449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6822","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6826","nonce":2261503449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6814","nonce":2261503449},{"type":"v1","addr":"172.21.15.136:6818","nonce":2261503449}]},"public_addr":"172.21.15.136:6803/2261503449","cluster_addr":"172.21.15.136:6811/2261503449","heartbeat_back_addr":"172.21.15.136:6826/2261503449","heartbeat_front_addr":"172.21.15.136:6818/2261503449","state":["exists","up"]},{"osd":7,"uuid":"c9e5e983-106f-4d01-a153-cdf7d656a091","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6802","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6804","nonce":1398734507}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6805","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6807","nonce":1398734507}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6812","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6814","nonce":1398734507}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6809","nonce":1398734507},{"type":"v1","addr":"172.21.15.142:6810","nonce":1398734507}]},"public_addr":"172.21.15.142:6804/1398734507","cluster_addr":"172.21.15.142:6807/1398734507","heartbeat_back_addr":"172.21.15.142:6814/1398734507","heartbeat_front_addr":"172.21.15.142:6810/1398734507","state":["exists","up"]},{"osd":8,"uuid":"a711d0fa-f5a7-4146-b280-8ee969afbb0f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6811","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6815","nonce":786826444}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6819","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6821","nonce":786826444}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6829","nonce":786826444}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":786826444},{"type":"v1","addr":"172.21.15.73:6826","nonce":786826444}]},"public_addr":"172.21.15.73:6815/786826444","cluster_addr":"172.21.15.73:6821/786826444","heartbeat_back_addr":"172.21.15.73:6829/786826444","heartbeat_front_addr":"172.21.15.73:6826/786826444","state":["exists","up"]},{"osd":9,"uuid":"914af14e-f90d-42de-918a-d925f50ccfb2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6815","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6817","nonce":2287493993}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6818","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6819","nonce":2287493993}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6822","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6823","nonce":2287493993}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6820","nonce":2287493993},{"type":"v1","addr":"172.21.15.120:6821","nonce":2287493993}]},"public_addr":"172.21.15.120:6817/2287493993","cluster_addr":"172.21.15.120:6819/2287493993","heartbeat_back_addr":"172.21.15.120:6823/2287493993","heartbeat_front_addr":"172.21.15.120:6821/2287493993","state":["exists","up"]},{"osd":10,"uuid":"6ad314a3-6c86-46fd-b1a6-d20911da6d24","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6800","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6802","nonce":1419708480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6805","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6808","nonce":1419708480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6821","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6825","nonce":1419708480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6812","nonce":1419708480},{"type":"v1","addr":"172.21.15.136:6817","nonce":1419708480}]},"public_addr":"172.21.15.136:6802/1419708480","cluster_addr":"172.21.15.136:6808/1419708480","heartbeat_back_addr":"172.21.15.136:6825/1419708480","heartbeat_front_addr":"172.21.15.136:6817/1419708480","state":["exists","up"]},{"osd":11,"uuid":"2ea8b70d-e122-4c29-8742-18d341fd8668","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6824","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6825","nonce":2871475141}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6826","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6827","nonce":2871475141}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6830","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6831","nonce":2871475141}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6828","nonce":2871475141},{"type":"v1","addr":"172.21.15.142:6829","nonce":2871475141}]},"public_addr":"172.21.15.142:6825/2871475141","cluster_addr":"172.21.15.142:6827/2871475141","heartbeat_back_addr":"172.21.15.142:6831/2871475141","heartbeat_front_addr":"172.21.15.142:6829/2871475141","state":["exists","up"]},{"osd":12,"uuid":"24086ac0-a92d-400b-9b3d-f48992dc8679","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":21,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6807","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6810","nonce":3885787537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6818","nonce":3885787537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6832","nonce":3885787537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":3885787537},{"type":"v1","addr":"172.21.15.73:6825","nonce":3885787537}]},"public_addr":"172.21.15.73:6810/3885787537","cluster_addr":"172.21.15.73:6818/3885787537","heartbeat_back_addr":"172.21.15.73:6832/3885787537","heartbeat_front_addr":"172.21.15.73:6825/3885787537","state":["exists","up"]},{"osd":13,"uuid":"da98eea5-e373-4325-adda-75f747ad6a99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6824","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6825","nonce":2634549424}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6826","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6827","nonce":2634549424}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6830","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6831","nonce":2634549424}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.120:6828","nonce":2634549424},{"type":"v1","addr":"172.21.15.120:6829","nonce":2634549424}]},"public_addr":"172.21.15.120:6825/2634549424","cluster_addr":"172.21.15.120:6827/2634549424","heartbeat_back_addr":"172.21.15.120:6831/2634549424","heartbeat_front_addr":"172.21.15.120:6829/2634549424","state":["exists","up"]},{"osd":14,"uuid":"d03df2d4-8d1c-4c2a-9719-d64ad780acc7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":22,"up_thru":72,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6806","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6810","nonce":381469963}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6813","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6816","nonce":381469963}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6828","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6829","nonce":381469963}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.136:6820","nonce":381469963},{"type":"v1","addr":"172.21.15.136:6824","nonce":381469963}]},"public_addr":"172.21.15.136:6810/381469963","cluster_addr":"172.21.15.136:6816/381469963","heartbeat_back_addr":"172.21.15.136:6829/381469963","heartbeat_front_addr":"172.21.15.136:6824/381469963","state":["exists","up"]},{"osd":15,"uuid":"7aa92378-d336-41be-85f1-2eb582f54947","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":22,"last_clean_end":48,"up_from":67,"up_thru":72,"down_at":49,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6800","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6801","nonce":1087448681}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6803","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6806","nonce":1087448681}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6813","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6815","nonce":1087448681}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.142:6808","nonce":1087448681},{"type":"v1","addr":"172.21.15.142:6811","nonce":1087448681}]},"public_addr":"172.21.15.142:6801/1087448681","cluster_addr":"172.21.15.142:6806/1087448681","heartbeat_back_addr":"172.21.15.142:6815/1087448681","heartbeat_front_addr":"172.21.15.142:6811/1087448681","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:10.978166+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:05.781059+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:07.537155+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:58.962904+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:06.855916+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:51.333967+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:15.149177+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:13.703389+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:11:59.025889+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:01.307602+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:09.979832+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:13.336889+0000","dead_epoch":0},{"osd":12,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:12.170064+0000","dead_epoch":0},{"osd":13,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:13:02.057520+0000","dead_epoch":0},{"osd":14,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:40.665511+0000","dead_epoch":0},{"osd":15,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-18T23:12:02.044564+0000","dead_epoch":49}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"},"shecprofile":{"c":"2","crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","k":"4","m":"3","name":"shecprofile","plugin":"shec","technique":"multiple"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-18T23:13:16.794 INFO:tasks.ceph:Scrubbing osd.0 2024-04-18T23:13:16.794 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:16.953 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:16.954 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:16.954 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:16.969 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-18T23:13:17.247 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 0 to deep-scrub 2024-04-18T23:13:17.265 INFO:tasks.ceph:Scrubbing osd.4 2024-04-18T23:13:17.265 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:17.414 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:17.414 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:17.414 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:17.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-18T23:13:17.719 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 4 to deep-scrub 2024-04-18T23:13:17.740 INFO:tasks.ceph:Scrubbing osd.8 2024-04-18T23:13:17.740 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.8 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:17.875 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:17.876 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:17.876 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:17.891 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 8 2024-04-18T23:13:18.158 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 8 to deep-scrub 2024-04-18T23:13:18.176 INFO:tasks.ceph:Scrubbing osd.12 2024-04-18T23:13:18.176 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.12 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:18.328 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:18.328 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:18.328 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:18.344 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 12 2024-04-18T23:13:18.624 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 12 to deep-scrub 2024-04-18T23:13:18.644 INFO:tasks.ceph:Scrubbing osd.1 2024-04-18T23:13:18.644 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:18.791 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:18.792 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:18.792 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:18.807 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-18T23:13:19.095 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 1 to deep-scrub 2024-04-18T23:13:19.114 INFO:tasks.ceph:Scrubbing osd.5 2024-04-18T23:13:19.115 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:19.261 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:19.261 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:19.261 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:19.277 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-18T23:13:19.565 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 5 to deep-scrub 2024-04-18T23:13:19.582 INFO:tasks.ceph:Scrubbing osd.9 2024-04-18T23:13:19.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.9 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:19.720 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:19.720 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:19.720 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:19.735 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 9 2024-04-18T23:13:20.024 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 9 to deep-scrub 2024-04-18T23:13:20.043 INFO:tasks.ceph:Scrubbing osd.13 2024-04-18T23:13:20.043 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.13 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:20.191 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:20.191 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:20.191 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:20.208 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 13 2024-04-18T23:13:20.486 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 13 to deep-scrub 2024-04-18T23:13:20.503 INFO:tasks.ceph:Scrubbing osd.2 2024-04-18T23:13:20.504 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:20.639 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:20.639 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:20.639 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:20.653 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-18T23:13:20.917 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 2 to deep-scrub 2024-04-18T23:13:20.934 INFO:tasks.ceph:Scrubbing osd.6 2024-04-18T23:13:20.934 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:21.076 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:21.076 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:21.076 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:21.091 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-18T23:13:21.363 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 6 to deep-scrub 2024-04-18T23:13:21.381 INFO:tasks.ceph:Scrubbing osd.10 2024-04-18T23:13:21.381 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.10 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:21.515 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:21.516 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:21.516 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:21.531 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 10 2024-04-18T23:13:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 10 to deep-scrub 2024-04-18T23:13:21.826 INFO:tasks.ceph:Scrubbing osd.14 2024-04-18T23:13:21.826 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.14 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:21.963 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:21.964 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:21.964 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:21.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 14 2024-04-18T23:13:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 14 to deep-scrub 2024-04-18T23:13:22.260 INFO:tasks.ceph:Scrubbing osd.3 2024-04-18T23:13:22.260 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:22.405 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:22.405 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:22.405 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:22.420 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-18T23:13:22.708 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 3 to deep-scrub 2024-04-18T23:13:22.727 INFO:tasks.ceph:Scrubbing osd.7 2024-04-18T23:13:22.727 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:22.876 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:22.876 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:22.876 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:22.892 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-18T23:13:23.168 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 7 to deep-scrub 2024-04-18T23:13:23.186 INFO:tasks.ceph:Scrubbing osd.11 2024-04-18T23:13:23.186 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.11 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:23.335 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:23.335 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:23.335 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:23.351 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 11 2024-04-18T23:13:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 11 to deep-scrub 2024-04-18T23:13:23.646 INFO:tasks.ceph:Scrubbing osd.15 2024-04-18T23:13:23.646 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.15 config set osd_debug_deep_scrub_sleep 0 2024-04-18T23:13:23.790 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2024-04-18T23:13:23.790 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-18T23:13:23.790 INFO:teuthology.orchestra.run.smithi073.stdout:} 2024-04-18T23:13:23.805 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 15 2024-04-18T23:13:24.090 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 15 to deep-scrub 2024-04-18T23:13:24.108 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:13:24.379 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:24.379 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:13:24.395 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":226,"stamp":"2024-04-18T23:13:23.311174+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":553724,"kb_used_data":39376,"kb_used_omap":35,"kb_used_meta":514076,"kb_avail":1509395716,"statfs":{"total":1546188226560,"available":1545621213184,"internally_reserved":0,"allocated":40321024,"data_stored":24095008,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":526414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":15,"apply_latency_ms":15,"commit_latency_ns":15000000,"apply_latency_ns":15000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001136"},"pg_stats":[{"pgid":"1.0","version":"33'157","reported_seq":494,"reported_epoch":160,"state":"active+clean+scrubbing+deep","last_fresh":"2024-04-18T23:13:21.405090+0000","last_change":"2024-04-18T23:13:21.404614+0000","last_active":"2024-04-18T23:13:21.405090+0000","last_peered":"2024-04-18T23:13:21.405090+0000","last_clean":"2024-04-18T23:13:21.405090+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:13:21.405090+0000","last_undegraded":"2024-04-18T23:13:21.405090+0000","last_fullsized":"2024-04-18T23:13:21.405090+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"33'157","last_scrub_stamp":"2024-04-18T23:13:05.784378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:41.351544+0000","last_clean_scrub_stamp":"2024-04-18T23:13:05.784378+0000","objects_scrubbed":0,"log_size":157,"log_dups_size":0,"ondisk_log_size":157,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"deep scrubbing for 0s","scrub_duration":303,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":301,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:20.762061+0000","last_change":"2024-04-18T23:13:20.506137+0000","last_active":"2024-04-18T23:13:20.762061+0000","last_peered":"2024-04-18T23:13:20.762061+0000","last_clean":"2024-04-18T23:13:20.762061+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:13:20.762061+0000","last_undegraded":"2024-04-18T23:13:20.762061+0000","last_fullsized":"2024-04-18T23:13:20.762061+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:20.505906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:20.505906+0000","last_clean_scrub_stamp":"2024-04-18T23:13:20.505906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:22.422564+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":284,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:21.294373+0000","last_change":"2024-04-18T23:13:11.072187+0000","last_active":"2024-04-18T23:13:21.294373+0000","last_peered":"2024-04-18T23:13:21.294373+0000","last_clean":"2024-04-18T23:13:21.294373+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:13:21.294373+0000","last_undegraded":"2024-04-18T23:13:21.294373+0000","last_fullsized":"2024-04-18T23:13:21.294373+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:11.071951+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:13:11.071951+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:25.724133+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":280,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:21.294648+0000","last_change":"2024-04-18T23:12:29.985926+0000","last_active":"2024-04-18T23:13:21.294648+0000","last_peered":"2024-04-18T23:13:21.294648+0000","last_clean":"2024-04-18T23:13:21.294648+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:13:21.294648+0000","last_undegraded":"2024-04-18T23:13:21.294648+0000","last_fullsized":"2024-04-18T23:13:21.294648+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:29.985705+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:29.985705+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":107,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":276,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:21.294383+0000","last_change":"2024-04-18T23:12:14.068230+0000","last_active":"2024-04-18T23:13:21.294383+0000","last_peered":"2024-04-18T23:13:21.294383+0000","last_clean":"2024-04-18T23:13:21.294383+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:13:21.294383+0000","last_undegraded":"2024-04-18T23:13:21.294383+0000","last_fullsized":"2024-04-18T23:13:21.294383+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:14.068006+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:14.068006+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:36.692265+0000","scrub_duration":435,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:21.391937+0000","last_change":"2024-04-18T23:13:18.452570+0000","last_active":"2024-04-18T23:13:21.391937+0000","last_peered":"2024-04-18T23:13:21.391937+0000","last_clean":"2024-04-18T23:13:21.391937+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:13:21.391937+0000","last_undegraded":"2024-04-18T23:13:21.391937+0000","last_fullsized":"2024-04-18T23:13:21.391937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:18.452387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:18.452387+0000","last_clean_scrub_stamp":"2024-04-18T23:13:18.452387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:38.278505+0000","scrub_duration":199,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":279,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:21.365169+0000","last_change":"2024-04-18T23:12:13.697974+0000","last_active":"2024-04-18T23:13:21.365169+0000","last_peered":"2024-04-18T23:13:21.365169+0000","last_clean":"2024-04-18T23:13:21.365169+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:13:21.365169+0000","last_undegraded":"2024-04-18T23:13:21.365169+0000","last_fullsized":"2024-04-18T23:13:21.365169+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.697732+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.697732+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"queued for deep scrub","scrub_duration":299,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":288,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:20.754468+0000","last_change":"2024-04-18T23:13:20.024963+0000","last_active":"2024-04-18T23:13:20.754468+0000","last_peered":"2024-04-18T23:13:20.754468+0000","last_clean":"2024-04-18T23:13:20.754468+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:13:20.754468+0000","last_undegraded":"2024-04-18T23:13:20.754468+0000","last_fullsized":"2024-04-18T23:13:20.754468+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:20.024829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:20.024829+0000","last_clean_scrub_stamp":"2024-04-18T23:13:20.024829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:26.931557+0000","scrub_duration":99,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":202,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:21.494435+0000","last_change":"2024-04-18T23:12:13.759380+0000","last_active":"2024-04-18T23:13:21.494435+0000","last_peered":"2024-04-18T23:13:21.494435+0000","last_clean":"2024-04-18T23:13:21.494435+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:13:21.494435+0000","last_undegraded":"2024-04-18T23:13:21.494435+0000","last_fullsized":"2024-04-18T23:13:21.494435+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:12:13.758998+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:10:44.375476+0000","last_clean_scrub_stamp":"2024-04-18T23:12:13.758998+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:13:41.357877+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":157,"ondisk_log_size":157,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808856,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30968,"kb_used_data":2408,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94340872,"statfs":{"total":96636764160,"available":96605052928,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280549,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34108,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337732,"statfs":{"total":96636764160,"available":96601837568,"internally_reserved":0,"allocated":2998272,"data_stored":1975950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313252,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33572,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338268,"statfs":{"total":96636764160,"available":96602386432,"internally_reserved":0,"allocated":2449408,"data_stored":1437422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313252,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33556,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338284,"statfs":{"total":96636764160,"available":96602402816,"internally_reserved":0,"allocated":2433024,"data_stored":1427822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313253,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33560,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338280,"statfs":{"total":96636764160,"available":96602398720,"internally_reserved":0,"allocated":2433024,"data_stored":1427822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,6,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313253,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33580,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338260,"statfs":{"total":96636764160,"available":96602378240,"internally_reserved":0,"allocated":2449408,"data_stored":1437422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313253,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313251,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313251,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33560,"kb_used_data":2376,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338280,"statfs":{"total":96636764160,"available":96602398720,"internally_reserved":0,"allocated":2433024,"data_stored":1427822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313252,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33572,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338268,"statfs":{"total":96636764160,"available":96602386432,"internally_reserved":0,"allocated":2449408,"data_stored":1437422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280549,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34092,"kb_used_data":2912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337748,"statfs":{"total":96636764160,"available":96601853952,"internally_reserved":0,"allocated":2981888,"data_stored":1966350,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280549,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51620,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320220,"statfs":{"total":96636764160,"available":96583905280,"internally_reserved":0,"allocated":2449408,"data_stored":1437422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280550,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33576,"kb_used_data":2392,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338264,"statfs":{"total":96636764160,"available":96602382336,"internally_reserved":0,"allocated":2449408,"data_stored":1437422,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280549,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:13:24.399 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-18T23:13:05.784378+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=13, tm_sec=5, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=13, tm_sec=15, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T23:13:24.399 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-18T23:13:11.071951+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=13, tm_sec=11, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=13, tm_sec=15, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T23:13:24.399 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-18T23:12:29.985705+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=12, tm_sec=29, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=13, tm_sec=15, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T23:13:24.399 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-18T23:12:14.068006+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=12, tm_sec=14, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=13, tm_sec=15, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T23:13:24.400 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-18T23:12:13.697732+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=12, tm_sec=13, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=13, tm_sec=15, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T23:13:24.400 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-18T23:12:13.758998+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=12, tm_sec=13, tm_wday=3, tm_yday=109, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=18, tm_hour=23, tm_min=13, tm_sec=15, tm_wday=3, tm_yday=109, tm_isdst=0) 2024-04-18T23:13:24.400 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-18T23:13:44.401 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-18T23:13:44.695 INFO:teuthology.orchestra.run.smithi073.stdout: 2024-04-18T23:13:44.696 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2024-04-18T23:13:44.712 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":236,"stamp":"2024-04-18T23:13:43.317028+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":16,"num_per_pool_osds":16,"num_per_pool_omap_osds":10,"kb":1509949440,"kb_used":553912,"kb_used_data":39568,"kb_used_omap":35,"kb_used_meta":514076,"kb_avail":1509395528,"statfs":{"total":1546188226560,"available":1545621020672,"internally_reserved":0,"allocated":40517632,"data_stored":24210208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":36223,"internal_metadata":526414465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003571"},"pg_stats":[{"pgid":"1.0","version":"160'159","reported_seq":501,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:21.710906+0000","last_change":"2024-04-18T23:13:21.711062+0000","last_active":"2024-04-18T23:13:21.710906+0000","last_peered":"2024-04-18T23:13:21.710906+0000","last_clean":"2024-04-18T23:13:21.710906+0000","last_became_active":"2024-04-18T23:10:42.681455+0000","last_became_peered":"2024-04-18T23:10:42.681455+0000","last_unstale":"2024-04-18T23:13:21.710906+0000","last_undegraded":"2024-04-18T23:13:21.710906+0000","last_fullsized":"2024-04-18T23:13:21.710906+0000","mapping_epoch":22,"log_start":"0'0","ondisk_log_start":"0'0","created":22,"last_epoch_clean":23,"parent":"0.0","parent_split_bits":0,"last_scrub":"160'159","last_scrub_stamp":"2024-04-18T23:13:21.710669+0000","last_deep_scrub":"160'159","last_deep_scrub_stamp":"2024-04-18T23:13:21.710669+0000","last_clean_scrub_stamp":"2024-04-18T23:13:21.710669+0000","objects_scrubbed":2,"log_size":159,"log_dups_size":0,"ondisk_log_size":159,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:36.645872+0000","scrub_duration":307,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,2],"acting":[13,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":301,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:20.762061+0000","last_change":"2024-04-18T23:13:20.506137+0000","last_active":"2024-04-18T23:13:20.762061+0000","last_peered":"2024-04-18T23:13:20.762061+0000","last_clean":"2024-04-18T23:13:20.762061+0000","last_became_active":"2024-04-18T23:11:33.711909+0000","last_became_peered":"2024-04-18T23:11:33.711909+0000","last_unstale":"2024-04-18T23:13:20.762061+0000","last_undegraded":"2024-04-18T23:13:20.762061+0000","last_fullsized":"2024-04-18T23:13:20.762061+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:20.505906+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:20.505906+0000","last_clean_scrub_stamp":"2024-04-18T23:13:20.505906+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:22.422564+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,15],"acting":[5,15],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":293,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:24.751270+0000","last_change":"2024-04-18T23:13:24.751327+0000","last_active":"2024-04-18T23:13:24.751270+0000","last_peered":"2024-04-18T23:13:24.751270+0000","last_clean":"2024-04-18T23:13:24.751270+0000","last_became_active":"2024-04-18T23:10:45.497336+0000","last_became_peered":"2024-04-18T23:10:45.497336+0000","last_unstale":"2024-04-18T23:13:24.751270+0000","last_undegraded":"2024-04-18T23:13:24.751270+0000","last_fullsized":"2024-04-18T23:13:24.751270+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:24.751100+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:24.751100+0000","last_clean_scrub_stamp":"2024-04-18T23:13:24.751100+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:37.085047+0000","scrub_duration":103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":288,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:22.784198+0000","last_change":"2024-04-18T23:13:22.784253+0000","last_active":"2024-04-18T23:13:22.784198+0000","last_peered":"2024-04-18T23:13:22.784198+0000","last_clean":"2024-04-18T23:13:22.784198+0000","last_became_active":"2024-04-18T23:10:45.591282+0000","last_became_peered":"2024-04-18T23:10:45.591282+0000","last_unstale":"2024-04-18T23:13:22.784198+0000","last_undegraded":"2024-04-18T23:13:22.784198+0000","last_fullsized":"2024-04-18T23:13:22.784198+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:22.784003+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:22.784003+0000","last_clean_scrub_stamp":"2024-04-18T23:13:22.784003+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:49.784329+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[13,5],"acting":[13,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":13,"acting_primary":13,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":285,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:23.838528+0000","last_change":"2024-04-18T23:13:23.838635+0000","last_active":"2024-04-18T23:13:23.838528+0000","last_peered":"2024-04-18T23:13:23.838528+0000","last_clean":"2024-04-18T23:13:23.838528+0000","last_became_active":"2024-04-18T23:10:45.495715+0000","last_became_peered":"2024-04-18T23:10:45.495715+0000","last_unstale":"2024-04-18T23:13:23.838528+0000","last_undegraded":"2024-04-18T23:13:23.838528+0000","last_fullsized":"2024-04-18T23:13:23.838528+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:23.838315+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:23.838315+0000","last_clean_scrub_stamp":"2024-04-18T23:13:23.838315+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:24.540288+0000","scrub_duration":203,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,12],"acting":[3,12],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":294,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:21.391937+0000","last_change":"2024-04-18T23:13:18.452570+0000","last_active":"2024-04-18T23:13:21.391937+0000","last_peered":"2024-04-18T23:13:21.391937+0000","last_clean":"2024-04-18T23:13:21.391937+0000","last_became_active":"2024-04-18T23:10:46.453605+0000","last_became_peered":"2024-04-18T23:10:46.453605+0000","last_unstale":"2024-04-18T23:13:21.391937+0000","last_undegraded":"2024-04-18T23:13:21.391937+0000","last_fullsized":"2024-04-18T23:13:21.391937+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:18.452387+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:18.452387+0000","last_clean_scrub_stamp":"2024-04-18T23:13:18.452387+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:38.278505+0000","scrub_duration":199,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":287,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:22.300691+0000","last_change":"2024-04-18T23:13:22.300809+0000","last_active":"2024-04-18T23:13:22.300691+0000","last_peered":"2024-04-18T23:13:22.300691+0000","last_clean":"2024-04-18T23:13:22.300691+0000","last_became_active":"2024-04-18T23:10:45.600434+0000","last_became_peered":"2024-04-18T23:10:45.600434+0000","last_unstale":"2024-04-18T23:13:22.300691+0000","last_undegraded":"2024-04-18T23:13:22.300691+0000","last_fullsized":"2024-04-18T23:13:22.300691+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:22.300468+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:22.300468+0000","last_clean_scrub_stamp":"2024-04-18T23:13:22.300468+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:31.807283+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":288,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:20.754468+0000","last_change":"2024-04-18T23:13:20.024963+0000","last_active":"2024-04-18T23:13:20.754468+0000","last_peered":"2024-04-18T23:13:20.754468+0000","last_clean":"2024-04-18T23:13:20.754468+0000","last_became_active":"2024-04-18T23:10:45.489710+0000","last_became_peered":"2024-04-18T23:10:45.489710+0000","last_unstale":"2024-04-18T23:13:20.754468+0000","last_undegraded":"2024-04-18T23:13:20.754468+0000","last_fullsized":"2024-04-18T23:13:20.754468+0000","mapping_epoch":25,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":26,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:20.024829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:20.024829+0000","last_clean_scrub_stamp":"2024-04-18T23:13:20.024829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:26.931557+0000","scrub_duration":99,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,6],"acting":[8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":212,"reported_epoch":160,"state":"active+clean","last_fresh":"2024-04-18T23:13:24.599060+0000","last_change":"2024-04-18T23:13:24.599060+0000","last_active":"2024-04-18T23:13:24.599060+0000","last_peered":"2024-04-18T23:13:24.599060+0000","last_clean":"2024-04-18T23:13:24.599060+0000","last_became_active":"2024-04-18T23:11:33.518952+0000","last_became_peered":"2024-04-18T23:11:33.518952+0000","last_unstale":"2024-04-18T23:13:24.599060+0000","last_undegraded":"2024-04-18T23:13:24.599060+0000","last_fullsized":"2024-04-18T23:13:24.599060+0000","mapping_epoch":67,"log_start":"0'0","ondisk_log_start":"0'0","created":25,"last_epoch_clean":68,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-18T23:13:24.598817+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-18T23:13:24.598817+0000","last_clean_scrub_stamp":"2024-04-18T23:13:24.598817+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-18T23:14:28.768461+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[15,13],"acting":[15,13],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":15,"acting_primary":15,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":9},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":118,"num_write":203,"num_write_kb":3460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":15,"up_from":67,"seq":287762808860,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":30960,"kb_used_data":2408,"kb_used_omap":12,"kb_used_meta":28531,"kb_avail":94340880,"statfs":{"total":96636764160,"available":96605061120,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12297,"internal_metadata":29216759},"hb_peers":[0,2,3,4,5,6,7,8,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":14,"up_from":22,"seq":94489280553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,3,4,5,6,7,8,10,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":13,"up_from":22,"seq":94489280553,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34108,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337732,"statfs":{"total":96636764160,"available":96601837568,"internally_reserved":0,"allocated":2998272,"data_stored":1975950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[2,4,5,6,7,10,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":21,"seq":90194313256,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,7,9,12,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":21,"seq":90194313256,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33588,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338252,"statfs":{"total":96636764160,"available":96602370048,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,8,11,12,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":21,"seq":90194313257,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33584,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338256,"statfs":{"total":96636764160,"available":96602374144,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,2,4,6,9,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":21,"seq":90194313257,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33588,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338252,"statfs":{"total":96636764160,"available":96602370048,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,7,10,11,12,13],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":21,"seq":90194313257,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,4,6,8,11,12,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":21,"seq":90194313255,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,4,6,7,9,13,14,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":11,"up_from":21,"seq":90194313255,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,2,4,5,6,8,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":12,"up_from":21,"seq":90194313256,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,8,11,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":22,"seq":94489280553,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":34112,"kb_used_data":2928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337728,"statfs":{"total":96636764160,"available":96601833472,"internally_reserved":0,"allocated":2998272,"data_stored":1975950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,3,6,7,8,9,11,12,13,15],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":22,"seq":94489280553,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51640,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":49214,"kb_avail":94320200,"statfs":{"total":96636764160,"available":96583884800,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":50395590},"hb_peers":[3,4,5,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":22,"seq":94489280554,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,3,4,6,8,9,10,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":22,"seq":94489280553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33596,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338244,"statfs":{"total":96636764160,"available":96602361856,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,8,10,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":22,"seq":94489280553,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33592,"kb_used_data":2408,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338248,"statfs":{"total":96636764160,"available":96602365952,"internally_reserved":0,"allocated":2465792,"data_stored":1447022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,3,7,8,9,11,12,13,14],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":12,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":13,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":15,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-18T23:13:44.714 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-18T23:13:45.044 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-18T23:13:45.044 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-18T23:13:45.044 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-18T23:13:45.044 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:45.127 INFO:tasks.ceph.osd.0:Stopped 2024-04-18T23:13:45.127 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-18T23:13:45.127 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:45.226 INFO:tasks.ceph.osd.4:Stopped 2024-04-18T23:13:45.226 DEBUG:tasks.ceph.osd.8:waiting for process to exit 2024-04-18T23:13:45.226 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:51.329 INFO:tasks.ceph.osd.8:Stopped 2024-04-18T23:13:51.329 DEBUG:tasks.ceph.osd.12:waiting for process to exit 2024-04-18T23:13:51.330 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:51.415 INFO:tasks.ceph.osd.12:Stopped 2024-04-18T23:13:51.415 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-18T23:13:51.415 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:51.499 INFO:tasks.ceph.osd.1:Stopped 2024-04-18T23:13:51.499 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-18T23:13:51.499 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:53.355 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2024-04-18T23:13:57.600 INFO:tasks.ceph.osd.5:Stopped 2024-04-18T23:13:57.600 DEBUG:tasks.ceph.osd.9:waiting for process to exit 2024-04-18T23:13:57.600 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:57.675 INFO:tasks.ceph.osd.9:Stopped 2024-04-18T23:13:57.676 DEBUG:tasks.ceph.osd.13:waiting for process to exit 2024-04-18T23:13:57.676 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:57.768 INFO:tasks.ceph.osd.13:Stopped 2024-04-18T23:13:57.768 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-18T23:13:57.768 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:57.846 INFO:tasks.ceph.osd.2:Stopped 2024-04-18T23:13:57.846 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-18T23:13:57.846 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:57.936 INFO:tasks.ceph.osd.6:Stopped 2024-04-18T23:13:57.937 DEBUG:tasks.ceph.osd.10:waiting for process to exit 2024-04-18T23:13:57.937 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:58.015 INFO:tasks.ceph.osd.10:Stopped 2024-04-18T23:13:58.015 DEBUG:tasks.ceph.osd.14:waiting for process to exit 2024-04-18T23:13:58.015 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:58.096 INFO:tasks.ceph.osd.14:Stopped 2024-04-18T23:13:58.096 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-18T23:13:58.096 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:58.173 INFO:tasks.ceph.osd.3:Stopped 2024-04-18T23:13:58.174 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-18T23:13:58.174 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:58.256 INFO:tasks.ceph.osd.7:Stopped 2024-04-18T23:13:58.256 DEBUG:tasks.ceph.osd.11:waiting for process to exit 2024-04-18T23:13:58.257 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:58.326 INFO:tasks.ceph.osd.11:Stopped 2024-04-18T23:13:58.326 DEBUG:tasks.ceph.osd.15:waiting for process to exit 2024-04-18T23:13:58.326 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:13:58.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 is failed for ~0s 2024-04-18T23:13:58.935 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2024-04-18T23:14:04.427 INFO:tasks.ceph.osd.15:Stopped 2024-04-18T23:14:04.428 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-18T23:14:04.428 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-18T23:14:04.428 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:14:04.441 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 is failed for ~6s 2024-04-18T23:14:04.458 INFO:tasks.ceph.mgr.y:Stopped 2024-04-18T23:14:04.459 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-18T23:14:04.459 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:14:04.514 INFO:tasks.ceph.mgr.x:Stopped 2024-04-18T23:14:04.514 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-18T23:14:04.514 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-18T23:14:04.514 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:14:04.533 INFO:tasks.ceph.mon.a:Stopped 2024-04-18T23:14:04.533 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-18T23:14:04.534 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:14:04.555 INFO:tasks.ceph.mon.b:Stopped 2024-04-18T23:14:04.555 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-18T23:14:04.555 INFO:teuthology.orchestra.run:waiting for 300 2024-04-18T23:14:04.580 INFO:tasks.ceph.mon.c:Stopped 2024-04-18T23:14:04.580 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-18T23:14:04.580 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'slow request' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | head -n 1 2024-04-18T23:14:04.598 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:14:04.598 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-18T23:14:04.811 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:14:04.811 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-18T23:14:04.987 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-8 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:14:04.988 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-8 2024-04-18T23:14:05.136 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-12 on ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:14:05.136 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-12 2024-04-18T23:14:05.295 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:14:05.295 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-18T23:14:05.482 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:14:05.483 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-18T23:14:05.705 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-9 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:14:05.705 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-9 2024-04-18T23:14:05.884 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-13 on ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:14:05.884 DEBUG:teuthology.orchestra.run.smithi120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-13 2024-04-18T23:14:06.077 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:14:06.077 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-18T23:14:06.311 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:14:06.311 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-18T23:14:06.478 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-10 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:14:06.478 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-10 2024-04-18T23:14:06.610 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-14 on ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:14:06.610 DEBUG:teuthology.orchestra.run.smithi136:> sync && sudo umount -f /var/lib/ceph/osd/ceph-14 2024-04-18T23:14:06.762 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:14:06.762 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-18T23:14:06.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:14:06.981 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-18T23:14:07.171 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-11 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:14:07.172 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-11 2024-04-18T23:14:07.348 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-15 on ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:14:07.349 DEBUG:teuthology.orchestra.run.smithi142:> sync && sudo umount -f /var/lib/ceph/osd/ceph-15 2024-04-18T23:14:07.513 INFO:tasks.ceph:Archiving mon data... 2024-04-18T23:14:07.514 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/data/mon.a.tgz 2024-04-18T23:14:07.515 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2024-04-18T23:14:07.521 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.uiOhBt6eXA 2024-04-18T23:14:07.521 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.uiOhBt6eXA 2024-04-18T23:14:07.713 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.uiOhBt6eXA 2024-04-18T23:14:07.776 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.uiOhBt6eXA is 643KB 2024-04-18T23:14:07.839 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.uiOhBt6eXA 2024-04-18T23:14:07.844 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/data/mon.b.tgz 2024-04-18T23:14:07.844 DEBUG:teuthology.orchestra.run.smithi120:> mktemp 2024-04-18T23:14:07.851 INFO:teuthology.orchestra.run.smithi120.stdout:/tmp/tmp.YFM8yJr6aE 2024-04-18T23:14:07.851 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.YFM8yJr6aE 2024-04-18T23:14:08.046 DEBUG:teuthology.orchestra.run.smithi120:> sudo chmod 0666 /tmp/tmp.YFM8yJr6aE 2024-04-18T23:14:08.111 DEBUG:teuthology.orchestra.remote:smithi120:/tmp/tmp.YFM8yJr6aE is 643KB 2024-04-18T23:14:08.175 DEBUG:teuthology.orchestra.run.smithi120:> rm -fr /tmp/tmp.YFM8yJr6aE 2024-04-18T23:14:08.180 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/data/mon.c.tgz 2024-04-18T23:14:08.180 DEBUG:teuthology.orchestra.run.smithi136:> mktemp 2024-04-18T23:14:08.187 INFO:teuthology.orchestra.run.smithi136.stdout:/tmp/tmp.IZN2o1MNwI 2024-04-18T23:14:08.187 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.IZN2o1MNwI 2024-04-18T23:14:08.347 DEBUG:teuthology.orchestra.run.smithi136:> sudo chmod 0666 /tmp/tmp.IZN2o1MNwI 2024-04-18T23:14:08.407 DEBUG:teuthology.orchestra.remote:smithi136:/tmp/tmp.IZN2o1MNwI is 462KB 2024-04-18T23:14:08.469 DEBUG:teuthology.orchestra.run.smithi136:> rm -fr /tmp/tmp.IZN2o1MNwI 2024-04-18T23:14:08.475 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-18T23:14:08.475 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-18T23:14:08.478 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-18T23:14:08.480 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-18T23:14:08.517 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-18T23:14:08.537 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-18T23:14:08.542 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-18T23:14:08.580 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-18T23:14:08.587 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-18T23:14:08.587 INFO:tasks.ceph:Archiving crash dumps... 2024-04-18T23:14:08.589 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi073/crash 2024-04-18T23:14:08.589 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-18T23:14:08.604 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi120/crash 2024-04-18T23:14:08.605 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-18T23:14:08.619 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi136/crash 2024-04-18T23:14:08.620 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-18T23:14:08.636 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi142/crash 2024-04-18T23:14:08.637 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-18T23:14:08.651 INFO:tasks.ceph:Compressing logs... 2024-04-18T23:14:08.651 DEBUG:teuthology.orchestra.run.smithi073:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-18T23:14:08.653 DEBUG:teuthology.orchestra.run.smithi120:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-18T23:14:08.664 DEBUG:teuthology.orchestra.run.smithi136:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-18T23:14:08.675 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29992.log 2024-04-18T23:14:08.675 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35584.log 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29992.log: gzip -5 --verbose -- 0.0% /var/log/ceph/ceph-client.admin.37851.log 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29992.log.gz 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32840.log 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35584.log.gz 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37851.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.37851.log.gz /var/log/ceph/ceph-client.admin.39533.log 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39299.log 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32840.log.gz 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39533.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36174.log 2024-04-18T23:14:08.677 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39533.log.gz 2024-04-18T23:14:08.680 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39299.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30619.log 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39299.log.gz 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36174.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30871.log 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36174.log.gz 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30619.log.gz 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38247.log 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30871.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30871.log.gz 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37687.log 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38247.log.gz 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38200.log 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37687.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37687.log.gz 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35100.log 2024-04-18T23:14:08.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32414.log 2024-04-18T23:14:08.682 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38200.log.gz 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35100.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35100.log.gz 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35643.log 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32414.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35046.log 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32414.log.gz 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35643.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36304.log 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35643.log.gz 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39273.log 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35046.log.gz 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37817.log 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36304.log.gz 2024-04-18T23:14:08.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37583.log 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39273.log.gz 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30047.log 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37817.log.gz 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37583.log.gz 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39429.log 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31634.log 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30047.log.gz 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39429.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39429.log.gz 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30500.log 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31634.log.gz 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38675.log 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30500.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30500.log.gz 2024-04-18T23:14:08.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35538.log 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31564.log 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38675.log.gz 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35538.log.gz 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38226.log 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31564.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35242.log 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31564.log.gz 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38226.log.gz 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30956.log 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37609.log 2024-04-18T23:14:08.690 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35242.log.gz 2024-04-18T23:14:08.691 DEBUG:teuthology.orchestra.run.smithi142:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-18T23:14:08.695 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-18T23:14:08.695 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-18T23:14:08.695 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-18T23:14:08.696 INFO:teuthology.orchestra.run.smithi120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-18T23:14:08.696 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.1.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.9.log 2024-04-18T23:14:08.696 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.13.log 2024-04-18T23:14:08.696 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-18T23:14:08.696 INFO:teuthology.orchestra.run.smithi120.stderr: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-18T23:14:08.696 INFO:teuthology.orchestra.run.smithi120.stderr:/var/log/ceph/ceph-osd.13.log: /var/log/ceph/ceph-mon.b.log: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-18T23:14:08.705 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-18T23:14:08.713 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-18T23:14:08.713 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.14.log 2024-04-18T23:14:08.713 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-18T23:14:08.713 INFO:teuthology.orchestra.run.smithi136.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-18T23:14:08.713 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-osd.14.log: /var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-18T23:14:08.713 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.10.log 2024-04-18T23:14:08.713 INFO:teuthology.orchestra.run.smithi136.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-osd.10.log: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-18T23:14:08.713 INFO:teuthology.orchestra.run.smithi136.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-18T23:14:08.743 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25310.log 2024-04-18T23:14:08.743 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25189.log 2024-04-18T23:14:08.743 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31980.log 2024-04-18T23:14:08.743 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25310.log: /var/log/ceph/ceph-client.admin.25189.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25310.log.gz 2024-04-18T23:14:08.743 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.30322.log 2024-04-18T23:14:08.743 INFO:teuthology.orchestra.run.smithi142.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25189.log.gz 2024-04-18T23:14:08.743 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/objectstore_tool.31980.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25062.log 2024-04-18T23:14:08.743 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.0.30322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25086.log 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.30322.log.gz 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25062.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.32727.log 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25062.log.gz 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25086.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25086.log.gz 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25165.log 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/objectstore_tool.32727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25237.log 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25165.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25165.log.gz 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25237.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.31233.log 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25237.log.gz 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24990.log 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/objectstore_tool.31233.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr:gzip/var/log/ceph/ceph-client.admin.24990.log: -5 --verbose -- /var/log/ceph/objectstore_tool.30486.log 2024-04-18T23:14:08.744 INFO:teuthology.orchestra.run.smithi142.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24990.log.gz 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24942.log 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/objectstore_tool.30486.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.11.log 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:gzip/var/log/ceph/ceph-client.admin.24942.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.25213.log 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24942.log.gz 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25038.log 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25213.log.gz 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25141.log 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25038.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25038.log.gz 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.15.log 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25141.log.gz 2024-04-18T23:14:08.745 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25261.log 2024-04-18T23:14:08.750 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-osd.15.log: 93.4% -- replaced with /var/log/ceph/objectstore_tool.31980.log.gz 2024-04-18T23:14:08.750 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-18T23:14:08.750 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25261.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25261.log.gz 2024-04-18T23:14:08.750 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25110.log 2024-04-18T23:14:08.758 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-mgr.x.log: 93.6% -- replaced with /var/log/ceph/objectstore_tool.31233.log.gz 2024-04-18T23:14:08.758 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25014.log 2024-04-18T23:14:08.758 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25110.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25110.log.gz 2024-04-18T23:14:08.758 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.24966.log 2024-04-18T23:14:08.758 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25014.log.gz 2024-04-18T23:14:08.758 INFO:teuthology.orchestra.run.smithi142.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25286.log 2024-04-18T23:14:08.759 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.24966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.24966.log.gz 2024-04-18T23:14:08.759 INFO:teuthology.orchestra.run.smithi142.stderr: 93.1% 93.4% -- replaced with /var/log/ceph/objectstore_tool.32727.log.gz 2024-04-18T23:14:08.759 INFO:teuthology.orchestra.run.smithi142.stderr: -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-18T23:14:08.759 INFO:teuthology.orchestra.run.smithi142.stderr:/var/log/ceph/ceph-client.admin.25286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25286.log.gz 2024-04-18T23:14:08.789 INFO:teuthology.orchestra.run.smithi142.stderr: 94.3% -- replaced with /var/log/ceph/objectstore_tool.30486.log.gz 2024-04-18T23:14:08.797 INFO:teuthology.orchestra.run.smithi120.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-18T23:14:08.824 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30956.log.gz 2024-04-18T23:14:08.824 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31319.log 2024-04-18T23:14:08.824 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36229.log 2024-04-18T23:14:08.824 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37609.log.gz 2024-04-18T23:14:08.824 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37661.log 2024-04-18T23:14:08.824 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31319.log.gz 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36229.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36229.log.gz 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36037.log 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37828.log 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37661.log.gz 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36037.log.gz 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31060.log 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37828.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33987.log 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37828.log.gz 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31060.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31060.log.gz 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30075.log 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35185.log 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33987.log.gz 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30075.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30075.log.gz 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29548.log 2024-04-18T23:14:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34675.log 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35185.log.gz 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39065.log 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29548.log.gz 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39169.log 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34675.log.gz 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39065.log.gz 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34172.log 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39169.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31814.log 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39169.log.gz 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34172.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34172.log.gz 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32322.log 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38623.log 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31814.log.gz 2024-04-18T23:14:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32322.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32322.log.gz 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33877.log 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38623.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36063.log 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38623.log.gz 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33877.log.gz 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36089.log 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37832.log 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36063.log.gz 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35449.log 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36089.log.gz 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37739.log 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37832.log.gz 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35449.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35449.log.gz 2024-04-18T23:14:08.827 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31893.log 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37739.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.8.log 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37739.log.gz 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31893.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31893.log.gz 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32327.log 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31601.log 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32327.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32327.log.gz 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39351.log 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30772.log 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31601.log.gz 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36011.log 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39351.log.gz 2024-04-18T23:14:08.828 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34919.log 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30772.log.gz 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31136.log 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36011.log.gz 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36985.log 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34919.log.gz 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36668.log 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31136.log.gz 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32446.log 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36985.log.gz 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36668.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34969.log 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36668.log.gz 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32446.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34038.log 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32446.log.gz 2024-04-18T23:14:08.829 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34247.log 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34969.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34969.log.gz 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33627.log 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34038.log.gz 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34247.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34247.log.gz 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36136.log 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33627.log.gz 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36959.log 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr:gzip/var/log/ceph/ceph-client.admin.36136.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.38253.log 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36136.log.gz 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36798.log 2024-04-18T23:14:08.830 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36959.log.gz 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39377.log 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38253.log.gz 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34357.log 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36798.log.gz 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39377.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29598.log 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39377.log.gz 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34582.log 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34357.log.gz 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29598.log.gz 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32125.log 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29600.log 2024-04-18T23:14:08.831 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34582.log.gz 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37427.log 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32125.log.gz 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29600.log: gzip -5 --verbose 0.0% -- /var/log/ceph/ceph-client.admin.34222.log -- replaced with /var/log/ceph/ceph-client.admin.29600.log.gz 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39585.log 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37427.log.gz 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34222.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36590.log 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34222.log.gz 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39585.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32782.log 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39585.log.gz 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36590.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32267.log 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36590.log.gz 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30526.log 2024-04-18T23:14:08.832 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32782.log.gz 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30630.log 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32267.log.gz 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30526.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36746.log 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30526.log.gz 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37245.log 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30630.log.gz 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35965.log 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36746.log.gz 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29622.log 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37245.log.gz 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37401.log 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35965.log.gz 2024-04-18T23:14:08.833 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29622.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29631.log 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29622.log.gz 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39507.log 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37401.log.gz 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29631.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38329.log 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29631.log.gz 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37167.log 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39507.log.gz 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37479.log 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38329.log.gz 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37167.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20250.log 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37167.log.gz 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37479.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38857.log 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37479.log.gz 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.tmp-client.admin.20250.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35318.log 2024-04-18T23:14:08.834 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20250.log.gz 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32532.log 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38857.log.gz 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39117.log 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35318.log.gz 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35759.log 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32532.log.gz 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39117.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33765.log 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39117.log.gz 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35759.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31988.log 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35759.log.gz 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39611.log 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33765.log.gz 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38263.log 2024-04-18T23:14:08.835 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31988.log.gz 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32028.log 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39611.log.gz 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39325.log 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38263.log.gz 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38805.log 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32028.log.gz 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39325.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34518.log 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39325.log.gz 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31780.log 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38805.log.gz 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34862.log 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34518.log.gz 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31780.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38701.log 2024-04-18T23:14:08.836 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31780.log.gz 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31226.log 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34862.log.gz 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36278.log 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38701.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38701.log.gz 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35534.log 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31226.log.gz 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32895.log 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36278.log.gz 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32576.log 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35534.log.gz 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32895.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31197.log 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32895.log.gz 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32576.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31132.log 2024-04-18T23:14:08.837 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32576.log.gz 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38257.log 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31197.log.gz 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31132.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30015.log 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31132.log.gz 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38257.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33808.log 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38257.log.gz 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30015.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39221.log 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30015.log.gz 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33808.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35076.log 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33808.log.gz 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39221.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30474.log 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39221.log.gz 2024-04-18T23:14:08.838 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31410.log 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35076.log.gz 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36486.log 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30474.log.gz 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33186.log 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31410.log.gz 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31945.log 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36486.log.gz 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36356.log 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33186.log.gz 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39661.log 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31945.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31945.log.gz 2024-04-18T23:14:08.839 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39637.log 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36356.log.gz 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30034.log 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39661.log.gz 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37833.log 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39637.log.gz 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30034.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36230.log 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30034.log.gz 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29616.log 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37833.log.gz 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34283.log 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36230.log.gz 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29616.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33671.log 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29616.log.gz 2024-04-18T23:14:08.840 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35878.log 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34283.log.gz 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36408.log 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33671.log.gz 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38279.log 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35878.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35878.log.gz 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30024.log 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36408.log.gz 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38279.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31628.log 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38279.log.gz 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33338.log 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30024.log.gz 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38961.log 2024-04-18T23:14:08.841 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31628.log.gz 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33338.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36642.log 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33338.log.gz 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29446.log 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38961.log.gz 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33451.log 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36642.log: /var/log/ceph/ceph-client.admin.29446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36642.log.gz 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29611.log 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29446.log.gz 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38205.log 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33451.log.gz 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37063.log 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29611.log.gz 2024-04-18T23:14:08.842 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34787.log 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38205.log.gz 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32601.log 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37063.log.gz 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30928.log 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34787.log.gz 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33396.log 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32601.log.gz 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38727.log 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30928.log.gz 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33396.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31108.log 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33396.log.gz 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38727.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31736.log 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-18T23:14:08.843 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38242.log 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31108.log.gz 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31736.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36850.log 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31736.log.gz 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38242.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37848.log 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38242.log.gz 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38883.log 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36850.log.gz 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37848.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33882.log 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37848.log.gz 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37297.log 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38883.log.gz 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38987.log 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33882.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33882.log.gz 2024-04-18T23:14:08.844 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35399.log 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37297.log.gz 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25186.log 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38987.log.gz 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35399.log.gz 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29627.log 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.25186.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36772.log 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25186.log.gz 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32683.log 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29627.log.gz 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36772.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30058.log 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36772.log.gz 2024-04-18T23:14:08.845 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29621.log 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32683.log.gz 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30058.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33721.log 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30058.log.gz 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29621.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29496.log 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29621.log.gz 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31234.log 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33721.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33721.log.gz 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39403.log 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29496.log.gz 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38753.log 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31234.log.gz 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33852.log 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39403.log.gz 2024-04-18T23:14:08.846 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34466.log 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38753.log.gz 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32239.log 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33852.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33852.log.gz 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34466.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31036.log 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34466.log.gz 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29420.log 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32239.log.gz 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34644.log 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31036.log.gz 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37842.log 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29420.log.gz 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25067.log 2024-04-18T23:14:08.847 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34644.log.gz 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31703.log 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37842.log.gz 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.25067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33476.log 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25067.log.gz 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31703.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31703.log.gz 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29522.log 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33476.log.gz 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36382.log 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29522.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31919.log 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29522.log.gz 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29619.log 2024-04-18T23:14:08.848 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36382.log.gz 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31919.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31919.log.gz 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34383.log 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29619.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31315.log 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr: 86.7% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29619.log.gz 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33267.log 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34383.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34383.log.gz 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37089.log 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31315.log.gz 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33267.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29931.log 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33267.log.gz 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37089.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.37089.log.gz -5 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.32504.log 2024-04-18T23:14:08.849 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29605.log 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29931.log: /var/log/ceph/ceph-client.admin.32504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29931.log.gz 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32504.log.gz 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38909.log 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29605.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25402.log 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29605.log.gz 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38909.log.gz 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31444.log 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.25402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30984.log 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.25402.log.gz 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34071.log 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31444.log.gz 2024-04-18T23:14:08.850 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33126.log 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30984.log.gz 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29614.log 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34071.log.gz 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33126.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.33126.log.gz 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30054.log 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38213.log 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29614.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29614.log.gz 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30054.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37843.log 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30054.log.gz 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38213.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.38213.log.gz --verbose 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr: -- /var/log/ceph/ceph-client.admin.37505.log 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34408.log 2024-04-18T23:14:08.851 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37843.log.gz 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30552.log 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37505.log.gz 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34408.log.gz 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37375.log 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32211.log 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30552.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30552.log.gz 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37375.log.gz 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31534.log 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39143.log 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32211.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32211.log.gz 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38831.log 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31534.log.gz 2024-04-18T23:14:08.852 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39143.log.gz 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32921.log 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39039.log 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38831.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38831.log.gz 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30841.log 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32921.log.gz 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39039.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.39039.log.gz 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.30448.log 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32085.log 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30841.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30841.log.gz 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30448.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39455.log 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30448.log.gz 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32085.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32085.log.gz 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37765.log 2024-04-18T23:14:08.853 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34836.log 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39455.log.gz 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31504.log 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37765.log.gz 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34836.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34066.log 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34836.log.gz 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36907.log 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31504.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31504.log.gz 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34066.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37531.log 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34066.log.gz 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36907.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36907.log.gz 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32976.log 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36564.log 2024-04-18T23:14:08.854 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37531.log.gz 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36616.log 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32976.log.gz 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36564.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.36564.log.gz --verbose 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr: -- /var/log/ceph/ceph-client.admin.35750.log 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37141.log 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36616.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36616.log.gz 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39091.log 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35750.log.gz 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37141.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37141.log.gz 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31867.log 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37713.log 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39091.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39091.log.gz 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31867.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37219.log 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31867.log.gz 2024-04-18T23:14:08.855 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37713.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37713.log.gz -5 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39013.log 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39559.log 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37219.log.gz 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35283.log 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39013.log.gz 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39559.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39559.log.gz 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31594.log 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33534.log 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35283.log.gz 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31594.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31594.log.gz 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33534.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33534.log.gz 2024-04-18T23:14:08.856 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37193.log 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36824.log 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/ceph-client.admin.37193.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37193.log.gz 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31474.log 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36824.log.gz 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35673.log 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37349.log 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31474.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31474.log.gz 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37839.log 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35673.log.gz 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38239.log 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37349.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37349.log.gz 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29628.log 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37839.log.gz 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr: 89.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-18T23:14:08.857 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38239.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32531.log 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38239.log.gz 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29628.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37834.log 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29628.log.gz 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32295.log 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32531.log.gz 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33928.log 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37834.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37834.log.gz 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32295.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29634.log 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32295.log.gz 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33928.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.33928.log.gz --verbose 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr: -- /var/log/ceph/ceph-client.admin.35476.log 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37831.log 2024-04-18T23:14:08.858 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29634.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29634.log.gz 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35476.log.gz 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30029.log 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32709.log 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37831.log.gz 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30029.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30029.log.gz -5 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr: --verbose -- /var/log/ceph/ceph-osd.12.log 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29629.log 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32709.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32709.log.gz 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.12.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34412.log 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35960.log 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29629.log.gz 2024-04-18T23:14:08.859 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36330.log 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34412.log.gz 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36512.log 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35960.log.gz 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36330.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30582.log 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36330.log.gz 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30875.log 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36512.log.gz 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30000.log 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30582.log.gz 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30875.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37816.log 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30875.log.gz 2024-04-18T23:14:08.860 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32183.log 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30000.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30000.log.gz 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37816.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37850.log 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37816.log.gz 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34013.log 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32183.log.gz 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37115.log 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37850.log.gz 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38779.log 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34013.log.gz 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31840.log 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37115.log.gz 2024-04-18T23:14:08.861 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38779.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36460.log 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38779.log.gz 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37635.log 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31840.log.gz 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36460.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32154.log 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36460.log.gz 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37635.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30422.log 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37635.log.gz 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32154.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37835.log 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32154.log.gz 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38649.log 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30422.log.gz 2024-04-18T23:14:08.862 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39195.log 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37835.log.gz 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33241.log 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38649.log.gz 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39195.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38234.log 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39195.log.gz 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30627.log 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33241.log.gz 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38234.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39247.log 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38234.log.gz 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35902.log 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30627.log.gz 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39247.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38308.log 2024-04-18T23:14:08.863 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39247.log.gz 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35902.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37838.log 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35902.log.gz 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37011.log 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38308.log.gz 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37838.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37838.log.gz 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30048.log 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37011.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37011.log.gz 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32626.log 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30048.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30042.log 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30048.log.gz 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31084.log 2024-04-18T23:14:08.864 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32626.log.gz 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30042.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30042.log.gz 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31084.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32380.log 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31084.log.gz 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34753.log 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33097.log 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32380.log: /var/log/ceph/ceph-client.admin.34753.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38935.log 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32380.log.gz 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34753.log.gz 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33097.log.gz 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32480.log 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38935.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36720.log 2024-04-18T23:14:08.865 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.38935.log.gz 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.32480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36933.log 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32480.log.gz 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29470.log 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36720.log.gz 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35159.log 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36933.log.gz 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38117.log 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29470.log: /var/log/ceph/ceph-client.admin.35159.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30629.log 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29470.log.gz 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.35159.log.gz 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38117.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38117.log.gz 2024-04-18T23:14:08.866 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37323.log 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34277.log 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30629.log.gz 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36178.log 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37323.log.gz 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37271.log 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34277.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34277.log.gz 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36178.log.gz 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31012.log 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37271.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33720.log 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37271.log.gz 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.31012.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34616.log 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.31012.log.gz 2024-04-18T23:14:08.867 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35849.log 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33720.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33720.log.gz 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34616.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.34616.log.gz --verbose 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr: -- /var/log/ceph/ceph-client.admin.33001.log 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38223.log 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35849.log.gz 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30071.log 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33001.log.gz 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30809.log 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.38223.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38223.log.gz 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30071.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37849.log 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30071.log.gz 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37037.log 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30809.log.gz 2024-04-18T23:14:08.868 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35710.log 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37849.log.gz 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37453.log 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37037.log.gz 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36434.log 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35710.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35710.log.gz 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36876.log 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37453.log.gz 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35425.log 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36434.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36434.log.gz 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36876.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.33559.log 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36876.log.gz 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.35425.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35425.log.gz 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36694.log 2024-04-18T23:14:08.869 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.33559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39481.log 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.33559.log.gz 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36694.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30695.log 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36694.log.gz 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.39481.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39481.log.gz 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29632.log 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36538.log 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30695.log.gz 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30036.log 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29632.log.gz 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.36538.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.36538.log.gz 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34548.log 2024-04-18T23:14:08.870 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30014.log 2024-04-18T23:14:08.871 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30036.log.gz 2024-04-18T23:14:08.871 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.34548.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37557.log 2024-04-18T23:14:08.871 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.34548.log.gz 2024-04-18T23:14:08.871 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29595.log 2024-04-18T23:14:08.871 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.30014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.30014.log.gz 2024-04-18T23:14:08.871 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.37557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.37557.log.gz 2024-04-18T23:14:08.871 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.29595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.29595.log.gz 2024-04-18T23:14:08.916 INFO:teuthology.orchestra.run.smithi073.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-18T23:14:09.064 INFO:teuthology.orchestra.run.smithi136.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-18T23:14:09.078 INFO:teuthology.orchestra.run.smithi120.stderr: 91.2% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-18T23:14:09.124 INFO:teuthology.orchestra.run.smithi142.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-18T23:14:09.172 INFO:teuthology.orchestra.run.smithi142.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.11.log.gz 2024-04-18T23:14:09.176 INFO:teuthology.orchestra.run.smithi120.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-18T23:14:09.293 INFO:teuthology.orchestra.run.smithi073.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.8.log.gz 2024-04-18T23:14:09.416 INFO:teuthology.orchestra.run.smithi073.stderr: 91.3% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-18T23:14:09.427 INFO:teuthology.orchestra.run.smithi136.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.10.log.gz 2024-04-18T23:14:09.442 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.15 has been restored 2024-04-18T23:14:09.450 INFO:teuthology.orchestra.run.smithi120.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.9.log.gz 2024-04-18T23:14:09.494 INFO:teuthology.orchestra.run.smithi073.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.12.log.gz 2024-04-18T23:14:09.583 INFO:teuthology.orchestra.run.smithi142.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-18T23:14:09.587 INFO:teuthology.orchestra.run.smithi142.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.15.log.gz 2024-04-18T23:14:09.588 INFO:teuthology.orchestra.run.smithi142.stderr: 2024-04-18T23:14:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:real 0m0.888s 2024-04-18T23:14:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:user 0m2.561s 2024-04-18T23:14:09.589 INFO:teuthology.orchestra.run.smithi142.stderr:sys 0m0.130s 2024-04-18T23:14:09.605 INFO:teuthology.orchestra.run.smithi073.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-18T23:14:09.607 INFO:teuthology.orchestra.run.smithi073.stderr: 2024-04-18T23:14:09.607 INFO:teuthology.orchestra.run.smithi073.stderr:real 0m0.951s 2024-04-18T23:14:09.607 INFO:teuthology.orchestra.run.smithi073.stderr:user 0m3.282s 2024-04-18T23:14:09.607 INFO:teuthology.orchestra.run.smithi073.stderr:sys 0m0.206s 2024-04-18T23:14:09.621 INFO:teuthology.orchestra.run.smithi136.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-18T23:14:09.658 INFO:teuthology.orchestra.run.smithi136.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.14.log.gz 2024-04-18T23:14:09.669 INFO:teuthology.orchestra.run.smithi120.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.13.log.gz 2024-04-18T23:14:09.670 INFO:teuthology.orchestra.run.smithi120.stderr: 2024-04-18T23:14:09.670 INFO:teuthology.orchestra.run.smithi120.stderr:real 0m1.004s 2024-04-18T23:14:09.670 INFO:teuthology.orchestra.run.smithi120.stderr:user 0m2.620s 2024-04-18T23:14:09.670 INFO:teuthology.orchestra.run.smithi120.stderr:sys 0m0.122s 2024-04-18T23:14:09.694 INFO:teuthology.orchestra.run.smithi136.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-18T23:14:09.695 INFO:teuthology.orchestra.run.smithi136.stderr: 2024-04-18T23:14:09.695 INFO:teuthology.orchestra.run.smithi136.stderr:real 0m1.010s 2024-04-18T23:14:09.695 INFO:teuthology.orchestra.run.smithi136.stderr:user 0m3.839s 2024-04-18T23:14:09.695 INFO:teuthology.orchestra.run.smithi136.stderr:sys 0m0.123s 2024-04-18T23:14:09.696 INFO:tasks.ceph:Archiving logs... 2024-04-18T23:14:09.696 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi073/log 2024-04-18T23:14:09.697 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-18T23:14:10.393 DEBUG:teuthology.misc:Transferring archived files from smithi120:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi120/log 2024-04-18T23:14:10.394 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-18T23:14:10.760 DEBUG:teuthology.misc:Transferring archived files from smithi136:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi136/log 2024-04-18T23:14:10.761 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-18T23:14:11.276 DEBUG:teuthology.misc:Transferring archived files from smithi142:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi142/log 2024-04-18T23:14:11.277 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-18T23:14:11.690 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-18T23:14:11.700 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-18T23:14:11.701 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-18T23:14:11.704 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-18T23:14:11.707 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-18T23:14:11.709 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-18T23:14:11.824 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-18T23:14:11.834 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-18T23:14:11.834 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T23:14:11.837 DEBUG:teuthology.orchestra.run.smithi120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T23:14:11.840 DEBUG:teuthology.orchestra.run.smithi136:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T23:14:11.842 DEBUG:teuthology.orchestra.run.smithi142:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-18T23:14:11.848 INFO:teuthology.orchestra.run.smithi073.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T23:14:11.849 INFO:teuthology.orchestra.run.smithi073.stdout:============================================================================== 2024-04-18T23:14:11.849 INFO:teuthology.orchestra.run.smithi073.stdout:*hv01.front.sepi 67.205.162.81 3 u 24 64 177 0.105 -0.550 0.436 2024-04-18T23:14:11.849 INFO:teuthology.orchestra.run.smithi073.stdout:+hv02.front.sepi 63.231.80.2 3 u 18 64 177 0.088 +0.538 0.593 2024-04-18T23:14:11.849 INFO:teuthology.orchestra.run.smithi073.stdout:+hv03.front.sepi 74.6.168.72 3 u 20 64 177 0.108 +0.328 0.681 2024-04-18T23:14:11.849 INFO:teuthology.orchestra.run.smithi073.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:14:11.850 INFO:teuthology.orchestra.run.smithi120.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T23:14:11.850 INFO:teuthology.orchestra.run.smithi120.stdout:============================================================================== 2024-04-18T23:14:11.851 INFO:teuthology.orchestra.run.smithi120.stdout:+hv01.front.sepi 67.205.162.81 3 u 20 64 177 0.127 -1.961 1.359 2024-04-18T23:14:11.851 INFO:teuthology.orchestra.run.smithi120.stdout:+hv02.front.sepi 63.231.80.2 3 u 19 64 177 0.059 -0.311 1.377 2024-04-18T23:14:11.851 INFO:teuthology.orchestra.run.smithi120.stdout:*hv03.front.sepi 74.6.168.72 3 u 21 64 177 0.081 -0.379 1.346 2024-04-18T23:14:11.851 INFO:teuthology.orchestra.run.smithi120.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:14:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T23:14:11.854 INFO:teuthology.orchestra.run.smithi136.stdout:============================================================================== 2024-04-18T23:14:11.854 INFO:teuthology.orchestra.run.smithi136.stdout:+hv01.front.sepi 67.205.162.81 3 u 13 64 177 0.223 -1.803 1.227 2024-04-18T23:14:11.854 INFO:teuthology.orchestra.run.smithi136.stdout:*hv02.front.sepi 63.231.80.2 3 u 23 64 177 0.146 -0.096 1.441 2024-04-18T23:14:11.854 INFO:teuthology.orchestra.run.smithi136.stdout:+hv03.front.sepi 74.6.168.72 3 u 18 64 177 0.207 -0.230 1.293 2024-04-18T23:14:11.854 INFO:teuthology.orchestra.run.smithi136.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:14:11.858 INFO:teuthology.orchestra.run.smithi142.stdout: remote refid st t when poll reach delay offset jitter 2024-04-18T23:14:11.859 INFO:teuthology.orchestra.run.smithi142.stdout:============================================================================== 2024-04-18T23:14:11.859 INFO:teuthology.orchestra.run.smithi142.stdout:+hv01.front.sepi 67.205.162.81 3 u 20 64 177 0.131 -1.624 1.008 2024-04-18T23:14:11.859 INFO:teuthology.orchestra.run.smithi142.stdout:+hv02.front.sepi 63.231.80.2 3 u 15 64 177 0.176 +0.028 0.922 2024-04-18T23:14:11.859 INFO:teuthology.orchestra.run.smithi142.stdout:*hv03.front.sepi 74.6.168.72 3 u 17 64 177 0.142 -0.058 0.863 2024-04-18T23:14:11.859 INFO:teuthology.orchestra.run.smithi142.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-04-18T23:14:11.860 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-18T23:14:11.869 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-18T23:14:11.869 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-18T23:14:11.879 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-18T23:14:11.891 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-18T23:14:11.901 INFO:teuthology.task.internal:Duration was 857.845411 seconds 2024-04-18T23:14:11.901 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-18T23:14:11.911 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-18T23:14:11.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-18T23:14:11.914 DEBUG:teuthology.orchestra.run.smithi120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-18T23:14:11.917 DEBUG:teuthology.orchestra.run.smithi136:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-18T23:14:11.919 DEBUG:teuthology.orchestra.run.smithi142:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-18T23:14:11.955 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-18T23:14:11.955 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2024-04-18T23:14:11.956 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-18T23:14:12.001 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi120.front.sepia.ceph.com 2024-04-18T23:14:12.002 DEBUG:teuthology.orchestra.run.smithi120:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-18T23:14:12.013 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi136.front.sepia.ceph.com 2024-04-18T23:14:12.013 DEBUG:teuthology.orchestra.run.smithi136:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-18T23:14:12.023 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi142.front.sepia.ceph.com 2024-04-18T23:14:12.023 DEBUG:teuthology.orchestra.run.smithi142:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-18T23:14:12.033 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-18T23:14:12.033 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-18T23:14:12.046 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-18T23:14:12.056 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-18T23:14:12.065 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-18T23:14:12.099 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-18T23:14:12.099 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-18T23:14:12.114 DEBUG:teuthology.orchestra.run.smithi120:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-18T23:14:12.124 DEBUG:teuthology.orchestra.run.smithi136:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-18T23:14:12.137 DEBUG:teuthology.orchestra.run.smithi142:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-18T23:14:12.198 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-18T23:14:12.210 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-18T23:14:12.210 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-18T23:14:12.220 DEBUG:teuthology.orchestra.run.smithi120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-18T23:14:12.232 DEBUG:teuthology.orchestra.run.smithi136:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-18T23:14:12.245 DEBUG:teuthology.orchestra.run.smithi142:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-18T23:14:12.258 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-18T23:14:12.269 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-18T23:14:12.273 DEBUG:teuthology.orchestra.run.smithi120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-18T23:14:12.277 DEBUG:teuthology.orchestra.run.smithi136:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-18T23:14:12.278 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2024-04-18T23:14:12.284 INFO:teuthology.orchestra.run.smithi120.stdout:kernel.core_pattern = core 2024-04-18T23:14:12.293 DEBUG:teuthology.orchestra.run.smithi142:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-18T23:14:12.301 INFO:teuthology.orchestra.run.smithi136.stdout:kernel.core_pattern = core 2024-04-18T23:14:12.317 INFO:teuthology.orchestra.run.smithi142.stdout:kernel.core_pattern = core 2024-04-18T23:14:12.339 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-18T23:14:12.343 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:14:12.344 DEBUG:teuthology.orchestra.run.smithi120:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-18T23:14:12.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:14:12.349 DEBUG:teuthology.orchestra.run.smithi136:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-18T23:14:12.371 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:14:12.371 DEBUG:teuthology.orchestra.run.smithi142:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-18T23:14:12.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-18T23:14:12.386 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-18T23:14:12.397 INFO:teuthology.task.internal:Transferring archived files... 2024-04-18T23:14:12.397 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi073 2024-04-18T23:14:12.398 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-18T23:14:12.420 DEBUG:teuthology.misc:Transferring archived files from smithi120:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi120 2024-04-18T23:14:12.421 DEBUG:teuthology.orchestra.run.smithi120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-18T23:14:12.445 DEBUG:teuthology.misc:Transferring archived files from smithi136:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi136 2024-04-18T23:14:12.445 DEBUG:teuthology.orchestra.run.smithi136:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-18T23:14:12.468 DEBUG:teuthology.misc:Transferring archived files from smithi142:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-18_21:00:02-rados-squid-distro-default-smithi/7662678/remote/smithi142 2024-04-18T23:14:12.468 DEBUG:teuthology.orchestra.run.smithi142:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-18T23:14:12.492 INFO:teuthology.task.internal:Removing archive directory... 2024-04-18T23:14:12.492 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-18T23:14:12.494 DEBUG:teuthology.orchestra.run.smithi120:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-18T23:14:12.499 DEBUG:teuthology.orchestra.run.smithi136:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-18T23:14:12.509 DEBUG:teuthology.orchestra.run.smithi142:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-18T23:14:12.536 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-18T23:14:12.548 INFO:teuthology.task.internal:Not uploading archives. 2024-04-18T23:14:12.549 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-18T23:14:12.558 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-18T23:14:12.558 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-18T23:14:12.561 DEBUG:teuthology.orchestra.run.smithi120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-18T23:14:12.563 DEBUG:teuthology.orchestra.run.smithi136:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-18T23:14:12.564 INFO:teuthology.orchestra.run.smithi073.stdout: 658632 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 18 23:14 /home/ubuntu/cephtest 2024-04-18T23:14:12.565 DEBUG:teuthology.orchestra.run.smithi142:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-18T23:14:12.567 INFO:teuthology.orchestra.run.smithi120.stdout: 658637 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 18 23:14 /home/ubuntu/cephtest 2024-04-18T23:14:12.569 INFO:teuthology.orchestra.run.smithi136.stdout: 658624 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 18 23:14 /home/ubuntu/cephtest 2024-04-18T23:14:12.587 INFO:teuthology.orchestra.run.smithi142.stdout: 658638 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 18 23:14 /home/ubuntu/cephtest 2024-04-18T23:14:12.588 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-18T23:14:12.599 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-18T23:14:12.627 INFO:teuthology.run:Summary data: description: rados/thrash-erasure-code-shec/{ceph clusters/{fixed-4 openstack} mon_election/connectivity msgr-failures/osd-dispatch-delay objectstore/bluestore-bitmap rados recovery-overrides/{more-async-recovery} supported-random-distro$/{ubuntu_latest} thrashers/careful thrashosds-health workloads/ec-rados-plugin=shec-k=4-m=3-c=2} duration: 857.8454113006592 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-18T23:14:12.627 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-18T23:14:12.715 INFO:teuthology.run:pass